search

projecthorus / radiosonde_auto_rx

Automatically Track Radiosonde Launches using RTLSDR
GNU General Public License v3.0
484 stars 124 forks source link

IMet-1-ABXN Serial Generation Isssues #254

Closed ZigiWalter closed 4 years ago

ZigiWalter commented 4 years ago

Hi,

I've bumped into IMet naming issues again - a single sonde being identified as multiple ones. I thought it's the varying frequency thingy again, but then I've spotted something strange in the log files - frame numbers are increased by 2 (snipped pasted below). If this is correct, then I guess that the power_on_time calculation goes wrong, which results in a new hash value.

We pick up IMETs on a daily basis here - but it's the fist time I've spotted such behavior... I couldn't think of a simple mitigation - any inputs are welcomed.

Thanks.

20:12:03Z,IMET-E2596815,3931,33.29208,35.68154,9238.0,-42.8,63.2,iMet,401.998,SATS 12,BATT 5.0 20:12:04Z,IMET-E2596815,3933,33.29219,35.68155,9245.0,-42.8,62.9,iMet,401.998,SATS 10,BATT 5.0 20:12:05Z,IMET-E2596815,3935,33.29227,35.68153,9253.0,-42.8,62.7,iMet,401.998,SATS 10,BATT 5.0 20:12:06Z,IMET-E2596815,3937,33.29233,35.68149,9259.0,-42.9,62.1,iMet,401.998,SATS 11,BATT 5.0

20:13:03Z,IMET-F46CA05C,4049,33.29655,35.67995,9640.0,-45.9,51.2,iMet,401.999,SATS 10,BATT 5.0 20:13:04Z,IMET-F46CA05C,4051,33.29666,35.67995,9646.0,-46.0,51.1,iMet,401.999,SATS 10,BATT 5.0 20:13:05Z,IMET-F46CA05C,4053,33.29673,35.67997,9653.0,-46.1,50.8,iMet,401.999,SATS 11,BATT 5.0 20:13:06Z,IMET-F46CA05C,4055,33.29678,35.67999,9660.0,-46.2,50.8,iMet,401.999,SATS 11,BATT 5.0

darksidelemm commented 4 years ago

Ahh this is an iMet-1-RS sonde - an older type. We encountered this issue early on in adding support for the iMet sondes, and it's commented on here: https://github.com/projecthorus/radiosonde_auto_rx/pull/146

In short the iMet-1-RS sondes advance the packet number twice every second, while the iMet-4 units advance one per second. It's probably possible to detect this difference, but at the time I had kind of hoped the iMet-1-RS sondes were on their way out (they are obsolete) and this wouldn't be as big of an issue. We 'fixed' it by latching the calculated serial number when a sonde is detected - i.e. don't re-calculate on every new frame. This doesn't help when different stations receive the same sonde at different times, but I had thought this was an edge case. I guess the reason a new serial number was calculated in your case was because the signal was lost.

I'll leave this one one open as a placeholder. It might be possible to use two sequential packets to detect if the sonde is an iMet-1 or iMet-4.

How often has this occurred? (i.e. how many iMet-1-RS sondes are still kicking around in someones stores!)

ZigiWalter commented 4 years ago

Thanks for the explanation! Only saw a single one of those so far. Maybe someone has opened an old box lying around with this model - will keep track and update.

Thanks, Zigi.

darksidelemm commented 4 years ago

Thanks! I'm pretty sure its possible to detect the type of sonde based on a few sequential packets, but if I don't have to do it, then I won't :-P

rs1729 commented 4 years ago

The old iMet-1-RS has 60kHz bandwidth, the iMet-4 15kHz. Maybe it is enough to collect a few frames to see, how much the frame-counter advances each second. But caution, the old iMet-1-RS sometimes repeats a time/position-packet, but the ptu/counter-packet advances: https://github.com/projecthorus/radiosonde_auto_rx/issues/94#issuecomment-473624360 (13:07:28) is missing, (13:07:29) twice (incl. position and crc). Could be because of rounding seconds, and for the same timestamp it repeats the position.

darksidelemm commented 4 years ago

Its probably one of the narrowband iMet-1-ABXN sondes then. I have one sitting on my desk now producing the following decoded output: 

(09:50:04)  lat: -34.720825°  lon: 138.692719°  alt: 77m  sats: 6  #  CRC: 333D - 333D [OK]
[233]  P:1007.12mb  T:23.06°C  U:40.74%  bat:5.8V  #  CRC: 8080 - 8080 [OK]

(09:50:05)  lat: -34.720825°  lon: 138.692719°  alt: 77m  sats: 6  #  CRC: 231C - 231C [OK]
[235]  P:1007.20mb  T:23.08°C  U:40.70%  bat:5.8V  #  CRC: 98EB - 98EB [OK]

(09:50:06)  lat: -34.720829°  lon: 138.692719°  alt: 77m  sats: 6  #  CRC: 681E - 681E [OK]
[237]  P:1007.21mb  T:23.07°C  U:40.78%  bat:5.8V  #  CRC: 3252 - 3252 [OK]

(09:50:07)  lat: -34.720833°  lon: 138.692719°  alt: 77m  sats: 6  #  CRC: F59C - F59C [OK]
[239]  P:1007.29mb  T:23.02°C  U:40.73%  bat:5.8V  #  CRC: F5C6 - F5C6 [OK]

(09:50:08)  lat: -34.720833°  lon: 138.692719°  alt: 77m  sats: 6  #  CRC: 0473 - 0473 [OK]
[241]  P:1007.32mb  T:23.01°C  U:40.76%  bat:5.8V  #  CRC: 65A6 - 65A6 [OK]

(09:50:09)  lat: -34.720833°  lon: 138.692719°  alt: 77m  sats: 6  #  CRC: 1452 - 1452 [OK]
[243]  P:1007.36mb  T:22.90°C  U:40.71%  bat:5.8V  #  CRC: 1DD5 - 1DD5 [OK]

Same behaviour as the sonde mentioned above, so it's probably that.

rs1729 commented 4 years ago

Maybe this new one does not repeat frames anymore. Didn't see this for iMet-4. I have a recording of a iMet-1-RS with ozone packets (so 3 packets per second), the frame counter is still incremented by 2. (in the manual it is called "packet number") Maybe you can consider only consecutive frames where the timestamp increases by 1 second. However I don't have enough data to know if there could be other exceptions, but maybe this does not happen with the new generation of iMets.

ZigiWalter commented 4 years ago

Spotted the second one of those from the same location. I think I can do some local workaround in the spirit of the above (stall the generation of the ID of 2-3 packet until the frame rate is known). However, if you think a proper fix belongs in the main repository - I'll wait to avoid conflicts.

Thanks.

rs1729 commented 4 years ago

Do you have an audio sample of this iMet? Or even better an IQ-sample? Also for unknown signals IQ-samples are much better than FM audio samples, because you have more information about bandwidth and modulation. There are ways to record just the down sampled IF band, then the file size is not much bigger than FM-audio.

ZigiWalter commented 4 years ago

no - didn't know it can help. Will try to capture (but might miss all day-time launches until the next weekend :-( ) Before I do some Googling - do you have a recommended way to capture IQ-sample (Windows/RPI)?

darksidelemm commented 4 years ago

I think it's pretty certain that it's an iMet-1-ABXN. I can get a sample from the unit I have sitting on a shelf here.

If you want, you could enable save_decode_auto (see here: https://github.com/projecthorus/radiosonde_auto_rx/blob/master/auto_rx/station.cfg.example#L340 ), which will save the audio from the sonde as its receiving. You will have to get in and grab the file out before it starts receiving another sonde though. Note that save_decode_iq won't work for the iMet sondes at the moment, as that decoder isn't set up to take IQ input.

rs1729 commented 4 years ago

@darksidelemm sorry, with "unknown signals" I was referring to another issue about signals in the 400-406 MHz band.

@ZigiWalter on windows, with sdrsharp you can record the "RAW" IF-band, if you choose RAW (i.e. no demodulation, just downsampled and filtered IF-IQ) in "Radio" and record the RAW/IQ-audio as 16 bit stereo audio (not baseband) (didn't find this feature in sdrconsole, but would help decoding IQ-audio). For recording it is best to choose the widest band/lowpass filter (I think 32kHz). The IF sample rate is the original downconverted by a power of 2, but at least 32kHz. So with rtldsdr 1024000 sample rate, you can record IF at 32kHz, 2 channels 16 bit, not much bigger than mono-audio. If the sample rate of the signal is wider than NFM, you can always record baseband IQ.

darksidelemm commented 4 years ago

@ZigiWalter I noted an odd callsign RS_iMet appear on the map today from your station (ZW3B). Having a look through the code I'm having trouble figuring out how such a callsign could have been passed through to the Habitat uploader. Have you modified any of the code?

Messing with the code is fine, but if you do so, please put something extra into the version field in autorx/init.py so I don't go hunting around for bugs that are not in the main codebase!

ZigiWalter commented 4 years ago

My fault , sorry. I have implemented the workaround described in https://github.com/projecthorus/radiosonde_auto_rx/issues/254#issuecomment-554660092, but managed to introduce a bug even in those few new lines of code. I've recklessly forgot to turn off uploads to Habitat, and was not aware of that version field you have mentioned (I can imagine some checksum function that would upload a "signature" of the *.py files as that "version" field, to help protect from culprits like myself :-) ). I apologize for the bother I've caused.

I didn't notice additional lunches from that site other than the two I've reported above. I've fixed the bug, and at least for "regular" iMets the frame rate calculation appears to be OK now - the callsigns are consistent with the ones generated by other stations.

BTW - coincidentally, I've just noticed a semi-related issue with iMet callsigns - a near by station picked-up two iMet sondes that were using the same frequency simultaneously. Since the callsign is calculated once per decoder, the two sondes were identified as a single one creating a very strange path in SondeHub. However, this is so rare, that I don't think it's a real concern.

Thanks.

darksidelemm commented 4 years ago

Going to close this for now. If more of these iMet-1-ABXN show up, please re-open this issue.