jbrzusto / find_tags

search raw data streams for patterns from registered tags
GNU General Public License v2.0
2 stars 0 forks source link

tag not active bug #30

Closed jbrzusto closed 7 years ago

jbrzusto commented 7 years ago

here:

 Running tag finder on receiver SG-0816BBBK0C16 boot session 16 (resumed)
   Error in safeSys(bcmd, quote = FALSE): 
Error 2 from 'LD_LIBRARY_PATH=/usr/local/lib/boost_1.60 
/sgm/bin/find_tags_motus --pulses_to_confirm=8 --frequency_slop=0.5 --min_dfreq=0
 --max_dfreq=12 --pulse_slop=1.5 --burst_slop=4 --burst_slop_expansion=1 --use_events 
--max_skipped_bursts=20 --default_freq=166.376  --resume --bootnum=16 --src_sqlite /sgm/cache
/motus_meta_db.sqlite /sgm/recv//SG-0816BBBK0C16.motus  2>&1  ':
   resumed successfully
   motusID = 24574=0xaaa100
   Graph::find: tag not active
jbrzusto commented 7 years ago

Tag 24574

Tag deployed: 01 April 2017 (nominal) Tag deployment info edited: 09 May 2017 Saved state: Tue Jun 6 20:38:25 GMT 2017 Last data record timestamp at state save: Sun Apr 16 00:34:08 GMT 2017

jbrzusto commented 7 years ago

Also here:


Running tag finder on receiver SG-3214BBBK6103 boot session 155 (resumed)

Error in safeSys(bcmd, quote = FALSE): Error 2 from
'LD_LIBRARY_PATH=/usr/local/lib/boost_1.60 /sgm/bin/find_tags_motus
--pulses_to_confirm=8 --frequency_slop=0.5 --min_dfreq=0
--max_dfreq=12 --pulse_slop=1.5 --burst_slop=4
--burst_slop_expansion=1 --use_events --max_skipped_bursts=20
--default_freq=166.376 --resume --bootnum=155 --src_sqlite
/sgm/cache/motus_meta_db.sqlite /sgm/recv//SG-3214BBBK6103.motus 2>&1
':

   find_tags_motus: --resume failed
   motusID = 18469=0x1aeb130
   Graph::find: tag not active
jbrzusto commented 7 years ago

This event list triggers the error when adding tag 18277, and has found tag 18469 in the tree, but inactive.

ts            tagID       event     
------------  ----------  ----------
1436313600.0  18469       1         
1438876800.0  18469       0         
1455140700.0  18277       1         

The tag DB:

tagID       tsStart       tsEnd       
----------  ------------  ------------
18469       1436313600.0  1438876800.0
18277       1455140700.0  1471267999.3
20622       1468195200.0  1556535348.5

However (mystery!) with tag 20622 not in the DB, the error is not triggered, even though in the first case, the error is triggered before tag 20622 is activated!

jbrzusto commented 7 years ago

The problem appears to be with ambiguity handling. The receiver DB tagAmbig table maintains a mapping of pairs, triplets, etc. of ambiguous tags to a unique (and negative) ambigID. This mapping is loaded when the tag finder runs, whether or not a session is being resumed.

When a removeTag event occurs, we check whether the tag has been proxied by an ambiguity ID and then either delete the tag directly from the graph (if not proxied), or reduce the proxy in the graph by the tag being deleted, leading to another proxy (or a different real tag, if the proxy is for just two tags).

Unfortunately, this means the persistent database table is used to decide whether a tag has been proxied, so that if a batch is ever reprocessed, or if a tag is deployed more than once and it is proxied during an earlier activation, then delTag won't behave correctly because it's not checking that the tag is currently proxied in the graph!

So we need to distinguish two uses of the tagAmbig object/table:

  1. maintain a unique mapping between unordered pairs, triplets, ... hextuplets of motus tag IDs and negative ambiguity IDs
  2. record whether a given motusTagID is represented by a proxy in the current graph for a nominal frequency.

The first use persists across batches, whether resumed or not. The second use persists only across resumption of a batch, and so needs to be serialized separately.