User Tools

Site Tools


alarm_analysis:resilient_data_transfer

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
alarm_analysis:resilient_data_transfer [2018/02/21 12:39] sualarm_analysis:resilient_data_transfer [2023/12/29 13:41] (current) su
Line 3: Line 3:
 ===== Overview ===== ===== Overview =====
  
-Before reading this guide, you should be familiar with [[Alarm Analysis Network Architecture]] document. This illustrates possible network installations including scenarios where Alarms & Events are collected from a militarized zone and transferred to another network for processing.+Before reading this guide, you should be familiar with [[alarm_analysis:alarm_analysis_network_architectures|Alarm Analysis Network Architecture]] document. This illustrates possible network installations including scenarios where Alarms & Events are collected from a militarized zone and transferred to another network for processing.
  
 In these scenarios, it is necessary to configure Data Core installations to act as a data relay. Moreover, if the connectivity between relays is unreliable, further configuration is required to guarantee resilient data transfer. In these scenarios, it is necessary to configure Data Core installations to act as a data relay. Moreover, if the connectivity between relays is unreliable, further configuration is required to guarantee resilient data transfer.
Line 22: Line 22:
  
   * A second process on the Data Core Node polls the local data repository for stored events (Big Data Source) and sends to...   * A second process on the Data Core Node polls the local data repository for stored events (Big Data Source) and sends to...
-    - a further data relay that awaits an Ack from the App Store Connect (TCP Out). If Ack is not received, data is resubitted.+    - a further data relay that awaits an Ack from the App Store Connect (TCP Out). If Ack is not received, data is resubmitted.
  
   * App Store Connect listens for incoming events (TCP In) and stores to a local data repository (Big Data Sink).   * App Store Connect listens for incoming events (TCP In) and stores to a local data repository (Big Data Sink).
Line 127: Line 127:
 ==== 5. Testing ==== ==== 5. Testing ====
  
-Assuming that the TCP Printer Stream configured above is listening to an active stream, we should see evidence of Alarm & Event messages being collected in the PIN and BN Big Data repositories.+Assuming that the TCP Printer Stream configured above is listening to an active Alarm & Event stream, we should see evidence of data store to the Big Data repositories on the PIN and BN
 + 
 +A quick test is to execute a URL search query. 
 + 
 +1. Log on to servers hosting Data Core Node 
 + 
 +Open web browser and enter 
 +<code> http://localhost:9200/_cat/indices/tcpprinterstream.evt_*?v </code> 
 + 
 +A "doc.count" greater than zero indicates Alarm & Event data is successfully stored. 
 + 
 +<code> 
 +health status index                          pri rep docs.count docs.deleted store.size pri.store.size  
 +green  open   tcpprinterstream.evtidx_201802             10            0    211.7kb        211.7kb  
 +</code> 
 + 
 +A "doc.count" field  
 + 
 +2. Log on to servers hosting App Store Connect 
 + 
 +Open web browser and enter 
 +<code> http://localhost:9200/_cat/indices/tcpin.evt_*?v </code> 
 + 
 +You should expect to see something like: 
 + 
 +<code> 
 +health status index                          pri rep docs.count docs.deleted store.size pri.store.size  
 +green  open   tcpin.evtidx_201802              2           10           10    211.7kb        211.7kb  
 +</code> 
 + 
 +A "doc.count" greater than zero indicates Alarm & Event data is successfully stored. 
 + 
 +A "docs.deleted" greater than zero indicates events are arriving on both fast and resilient streams. The data consolidation process marks a duplicate data document for deletion. This is an active measure and clears to zero over time. 
 + 
 + 
 + 
 + 
 + 
 + 
 + 
  
  
alarm_analysis/resilient_data_transfer.1519216740.txt.gz · Last modified: 2018/02/21 12:39 by su