tag finder sessions for some non-166.38 MHz receivers are not using the correct listening frequency

[zcrysler]

I think there's something odd going on with some of the European receivers

Eg. proj 113, receiver SG-5016BBBK17E4 is in the Netherlands at 150.1 MHz BUT it has detections for SNPL which were deployed in New Mexico at 166.380 MHz. This receiver has never been registered anywhere else.

This isn't the only case, the same thing is seen on SG-5016BBBK23B1, Sg-5016BBBK1687, SG-5016BBBK23BI, SG-5016BBBK18D2, SG-5016BBBK17D2 all in project 113

Oddly, SG-5016BBBK0F85 (proj 113) has detections of New Mexico SNPL at 166.380 AND BrustHELGO tags at 150.100

Same is happening on proj 123 with German deployments eg. SG-3615BBBK0C15, and 76 eg. SG-1215BBBK1773

None of these receivers have data on the new server, this is based off of hourly plots, and project 113 reporting zero detections for their tags

[StuMackenzie]

If these were purchased from Compudata, they presumably would have been set and tested on 166.380MHz. Users would have had to reset all funcubes to 150.1 MHz themselves, which one would hope they would have realized during testing/station configuration.

[zcrysler]

That's my fear, want to be absolutely sure before telling people that they've lost a season of data

[StuMackenzie]

I sent a note to Compudata to confirm what frequency they would have shipped at - I assume 166.380. Then depending on what John thinks we can fask them if they tuned/tested their funcubes ahead of time...

[StuMackenzie]

All compudata units have been shipped at 166.380. There were a number that did not update correctly due to an earlier firmware issue - they didn't ID and resolve this issue until November, 2017.

[zcrysler]

SG issue to be corrected by users

[jbrzusto]

The listening frequency is a software setting, not stored in the funcube. The SG uses the value in the deployment.txt file to set the frequency, and records parameter-setting events in the data files, each time the SG is booted and any time the funcubes need to be restarted, (e.g. due to stalling, which is rare but happens).

As far as I've been able to tell, all users of tags at frequencies different from 166.38 MHz have been correctly setting their deployment.txt file correctly.

The problem is arising on the processing side: the tag finder reads antenna frequency settings from data files, and uses those to determine which tags to look for, based on the nominal tag frequency in the master motus tag database.

Frequency settings normally appear in the SG data files once per boot session for each antenna. However, sometimes they don't appear to make it into the data files. In that case, the tag finder uses a default frequency of 166.376 for each antenna it can't find a setting for, until it does find a setting in the data stream. For some long boot sessions where the tag finder has not seen a frequency setting, it will be processing a lot of data at the wrong frequency (i.e. looking for the wrong set of tags).

The workaround on the data server has been to create "parameter overrides" based on project ID, receiver serial number, and so on. e.g. a record in this table might tell the tag finder to assume a default antenna frequency of 150.396 MHz if processing data from a deployment under project X.

The parameter overrides are stored in a database, but there is currently no motus-side way of allowing users to provide this information. That's maybe not even necessary as it should really be a fix to the SG software: the SG should record a copy of deployment.txt in raw data files. But there are other reasons for overriding default tag-finder parameters, so ultimately it will be helpful to have a motus-side way of specifying these.

For now, I'm not sure exactly where the bug is in those sessions from 150 MHz receivers that are "detecting" tags at 166.376, but it is somewhere in the above scheme.

I've renamed this issue to reflect the underlying problem.

[StuMackenzie]

Thanks for the clarification John

[jbrzusto]

I didn't make it clear, but the upshot is that I don't think anyone will have lost any data to this, it's just that we haven't correctly found them yet...

[zcrysler]

Phew ok, so if it's unrelated to the compudata issue I'll hold off on telling them anything until the right frequency has been searched

[zcrysler]

Sander with project 113 needs to report to their funders soon but has no data because of the frequency issue - can you please let me know asap once you're able to sort it out?

[jbrzusto]

I have specified a default parameter override for project 113 and rerun each receiver. Here is their status:

Serno	Data	Last Connection to server (GMT)	Site
SG-5016BBBK0DC0	Yes	2018-02-09 17:19:34	ECN-003-2017
SG-5016BBBK0F73	Yes	2017-11-07 11:22:32	Cam-008-2017
SG-5016BBBK0F85	Yes	2017-11-07 11:49:07	Camp-009-2017
SG-5016BBBK1123	Yes	2018-02-09 17:23:27	IJmuiden - WURSG011
SG-5016BBBK1687	Yes	2017-11-07 13:51:57	Zwan-001.2017
SG-5016BBBK17D2	Yes	2017-11-03 10:38:02	Pett-007-2017
SG-5016BBBK17E4	Yes	2017-11-03 09:53:05	Pett-006-2017
SG-5016BBBK18D2	Yes	2018-02-09 17:20:28	Cast-005-2017
SG-5016BBBK1B6A	Yes	2017-11-07 14:18:47	Zwan.002-2017
SG-5016BBBK23B1	Yes	2018-02-09 17:23:28	Schagerbrug - 004-2017

Summary plots and data are available here: https://sgdata.motus.org/download/113

[jbrzusto]

For project 123, receiver SG-3615BBBK0C15's deployment record was added this morning, after data for that receiver had already been run, so the parameter override in place for that project was not used. I'm re-running that receiver.

[jbrzusto]

For project 76, SG-1215BBBK1773, the problem seems to have been jbrzusto/motusServer#350 so I'm rerunning that one again.

[zcrysler]

project 113 looks great, just want to make sure you're aware that for projects 123 and 76, it's more than just the receivers you listed above that are showing up at 166.380

[jbrzusto]

I'm rerunning all receivers from projects 123 and 76.

[jbrzusto]

The rerun has completed. Some summary plots still show a few tags at the 166.38 MHz, because the previous batches run with the wrong default listening frequency have not been removed from the master database yet. There is an open issue to deal with this situation: jbrzusto/TO_DO#138

[jbrzusto]

Re-running SG-4414BBBK0722, for which many data files had not been moved from the old to the new server.