Better handling of packets received in 'UDP mode'.

[darksidelemm]

UDP input mode was added mainly for testing purposes, but it's being used by some as a means of using auto_rx's packet processing and upload abilities. It's good to see auto_rx used in different ways, but we need to be sure that we don't break compatibility with other users when doing so!

UDP mode (starting auto_rx with --mode UDP --freq 401.0) allows auto_rx to accept the JSON that is normally emitted by the various decoder software on stdout, via a UDP socket. There are limitations in using this mode:

• The receiver frequency will not be available.
• There will be no SNR information available (this is only provided by the fsk_demod chains at the moment anyway)
• Currently there is a lot of processing that is dependent on the 'type' field which is passed around auto_rx to be valid. At the moment the type would be set to 'UDP', which will break a lot of downstream processing. The exception is the RS41 and DFM sondes which replace the 'type' field with a 'subtype' supplied by the decoder.

The following changes are going to need to be made for UDP input mode to be safe to use:

• Modify auto_rx to handle the lack of a receiver frequency in a more graceful way. At the moment whatever dummy frequency is used on startup is used.
• All decoders need to output a 'type' field. This can be hardcoded in most cases (e.g. 'type': 'LMS6'). This will then be used to direct further processing of the packets.
• Outputs from the multi-decoders need to be aligned with the current regular demod decoders, e.g. the recent M10 additions, and DFM battery output changes. (I need to submit a PR for these).
• In the case of the @rs1729 multi-decoders, it would be good if a 'freq' field could be output, but this does require the multi-decoder to have knowledge of the receiver centre frequency which may not always be the case.

[rs1729]

dfm09mod.c: I added the battery to the json output. It is only for DFM-09/STM32. If the type changes, there shouldn't be output for DFM-06.

m10: As long as it is experimental, the output should be on -vvv level. M10-PTU, it's been a while. If I'm not wrong, you added the humidity from m10ptu.c? Do you want it also in m10mod/json/sondehub? (And if so, maybe humidity should be grouped with temperature.) When I looked at relative humidity, often the data fit well and sometimes not; I was not satisfied. Mostly it was within 10-15%, but occasionally it was quite off. If it were a sensor failure, the station would be off, too. Ok, sometimes also the station had bad data.

Do you know if someone is using the multi-decoders? Forgot to add the DFM-json-battery-output there. Also the new soft decoding I didn't add to rs_multi yet. Though Hamming soft decoding is not for reliable json-frames anyway. ((For LMS6 I did a separate soft version, needs more memory and I wanted to test a bit more. First I was confused, since the FM-audio didn't play well with soft decoding. But when using IQ-data, it improved the decoding. With your test-samples (did only 1 test...) I got 8 OK-frames vs 1 with hard decoding at 5dB: lms6Xmod_soft --vit2 --ecc -v --IQ 0.0 - 96000 32 generated/lms6-400_96k_float_05.0dB.bin Between 5-6 dB Reed-Solomon got also something to do. Above 6 dB there was no difference between hard and soft decoding. Convolutional code has good performance. (Have to look at the SNR in the frequency spectrum, how it compares to RS41.) Weak FM-signals offen have spikes and the noise is not the same after FM-demodulation, noise at higher frequencies is problematic for FM. Narrow filtering before FM-demodulation is important. remark: I think on your test-page you still don't treat the Manchester codes the same, for RS92SGP you have 2400 baud, but the symbol rate is 4800. For DFM and M10 you have the raw baud rates. Maybe it would be fair for DFM and M10 to consider the bitrate as you did for RS92SGP. With Manchester you can integrate the bit over two symbols, Manchester helps synchronization, but the frequency spectrum gets wider, you could as well just transmit at slower speed (probably even better, if synchronization is no issue). Or you test only the raw symbol detection, but if you check the packet-error-rate, don't mix in the error correction. And I don't know if you consider the high modulation index for DFM, wider spectrum, but gives better demodulation results.)) Back to the multi-decoders: For RS41 and regular stations it could be useful. There is also the test-branch where you can add and remove signals. I didn't do much since then with the multi-branch. Maybe I should check first if there are some updates needed compared to the mod-decoders.

What about the json "type", what would be the difference to "subtype"? For RS41 e.g., would be "type" always "RS41" and "subtype" e.g. "RS41-SGP"? And for DFMs the type would be "DFM"? And the "type" would be for all radiosonde types, not only RS41 and DFM which have the "subtype"?

For RS41 the rs41-frequency could be added in json. But maybe you want the receiver frequency as well, since this would apply to all types.

[darksidelemm]

Hi @rs1729 !

The changes made to the DFM and M10 decoders are here: https://github.com/projecthorus/radiosonde_auto_rx/commit/b9f9ffb10bc9886c1a73e67f097786663fbf3335 I'm unsure how accurate the humidity values are, but it's somewhat better than no data at all!

Yes, a static 'type' field on the output for each decoder type just as you suggest would be perfect. "RS41" for RS41, "DFM" for DFM, "M10" for M10, "LMS6" for LMS6, etc... Sub-type is fine exactly as you have it now, I make use that already (e.g. "RS41-SGP" and the raw type code for the DFM sondes which I interpret.

I'll look into the modem testing stuff when I get a chance - we're currently in the process of improving the frequency estimation part of it, so I'll re-run the tests after that.

And yes, there are some users using the multi-decoder and feeding the JSON output into auto_rx via UDP - hence this issue!

[rs1729]

Step 1: added "type" to JSON output: https://github.com/rs1729/RS/commit/aab495085386ea626ce02f87c6a824d028893450 (also added "subtype" in lms6X, although it is also in the "id", but often the "type" is also in the "id".) The LMS6 type could also be just "LMS" analogous to "DFM"!? Don't know the latest status of these "X"-type test? sondes. They had the nicer frame structure.

Hope it's ok. Next I would sync the json-output of the multi-decoder, and look at ptu in m10ptu.c for m10mod.

Then we can think about frequency information in json-output.

[darksidelemm]

The type fields look good to me. I'll have to make some changes to auto_rx to use them, but I'll try and do that this weekend.

[rs1729]

Updated the json output of multi and the test-branch. Updated also the mk2a json output. Changed the json-type of the LMS-radiosondes to "LMS", the subtypes are: "6": LMS6-403 "X": lms6X "MK2A": LMS6-1680/Mk2a

[rs1729]

step 3: added (experimental) humidity (from m10ptu.c) to m10mod.c. relative humidity output with option --ptu and -vv (same for --json) and only for OK-frames.

How about the battery status provided by F5MVO, should that be added to? Is it ok with F5MVO? (added now; I also tried to relate the conversion factor to the ADC settings and PCB.)

[rs1729]

@Viproz @darksidelemm I was reading your highaltitude chat about M10+ (gtop) vs M10. The modulation is the same (I believe the PTU as well). Only the GPS data changed, when they replaced the Trimble with Gtop. We had this discussion before. It looked like it was only an intermediate version, for a short period of time. Probably they finally got a custom GPS sentence that resembled the Trimble data, when they moved on to the Gtop-successor SierraWireless. Nevertheless I included now the M10+ from rs1729/RS/m10/m10gtop.c, so both can be decoded with m10mod.c. In the json output I included the "subtype": "0x9F" (M10) "0xAF" (M10+) It is the second byte of the frame (after the first byte containing the frame length).

The M20 has also the same modulation except that the baud rate is rather 9600 than 9616. The frame data is a bit different, but I don't want to include it in the same module (like for lms6X), not only because of the baud difference. However dft_detect could read the first couple of bytes after detecting the sync-header, the second byte gives the M10/M20-type. I'll look at dft_detect.

[Viproz]

@rs1729 Awesome thanks, they are sending them 2 by two in Toulouse it seems like but I got the IDs later on and they are from 2017 so it probably is just some old stock they had laying around. I'll keep an eye out for some M20 according to an OM they try out the new RS in Toulouse before sending them to other launch site in France.

About the battery status it should be ok, he gave me the formula back then to use on the M10 decoder I made. It's been a while since I've seen him on IRC but if you want to check things with him I can give you his email I'm sure he won't mind.

[rs1729]

@Viproz Do you have the serial numbers of these M10+? Do they have the format "YMM 2 2xxxx", i.e. is the last part starting with "2...."?

Regarding battery, he contacted me already, no problem.

[Viproz]

Yes they are the normal format, F4IKV sent me the ids of the one he decoded: digitalsonde2020051912Z_M10new-704-2-22598 digitalsonde2020051912Z_M10new-704-2-22695 digitalsonde2020051518Z_M10new-704-2-22746 digitalsonde2020051512Z_M10new-704-2-22759 digitalsonde2020051512Z_M10new-704-2-22716

Same format as the ones they used to launch in Trappes (Paris) around 2 years ago.

[rs1729]

This would indicate once more that the first digit of the last part labels the (M10-)subtype, i.e. 1xxxx for M10 and 2xxxx for M10+.

The M20 has a different frame structure. If we assume that the SN is in the frame, the bits of the last part of the SN from Vienna can be found in the 2 bytes before the counter. (The M10-SN was in the bits before the counter, the M20-counter moved and there is an additional checksum the rounds up the block at the beginning that contains the important data.) The first part (YMM) could be at a different position. From only one example+SN-picture it is hard to say. But there will be more launches.

Update: Counting the 120 month of a decade in a consecutive way would need only 7 bits; probably it is contained in the remaining third byte. More samples would be nice.

[darksidelemm]

UDP input is much improved in https://github.com/projecthorus/radiosonde_auto_rx/releases/tag/v1.3.2

[rs1729]

Added json-field "freq" to the test branch: https://github.com/rs1729/RS/tree/test https://github.com/rs1729/RS/commit/8a088d4584ac404ea054e450d27b7e5e44192b6a "freq": frequency in kHz as integer. To see the "freq" in the json output, the decoder needs the (center) frequency of the signal/data or receiver in Hz via option --jsn_cfq <freq_Hz>, (center frequency for rs_multi or option --IQ <fq> in mod-decoders)

e.g. 5MHz IQ-data, center frequency 404MHz, M10 at 402.2MHz: ./m10mod -c -vv --json --jsn_cfq 404000000 --IQ -0.36 404000kHz_iq.wav or

./rs_multi --json --jsn_cfq 404000000 --m10 -0.36 404000kHz_iq.wav

<0: s=+0.9305, f=-0.3600>  Mon 2019-08-26 20:45:20.000  lat: 38.97495  lon: -77.07316  alt: 1880.51   vH: 5.9  D: 324.5  vV: -3.6   SN: 811 2 11760  #  [OK]  T=12.5C _RH=51%
{ "type": "M10", "frame": 1250887520 ,"id": "M10-811-2-11760", "datetime": "2019-08-26T20:45:02.000Z", "lat": 38.97495, "lon": -77.07316, "alt": 1880.51000, "vel_h": 5.90771, "heading": 324.50742, "vel_v": -3.64000, "sats": 9, "aprsid": "ME8B226E0", "batt": 5.06, "temp": 12.5, "humidity": 51.2, "subtype": "0x9F", "freq": 402200 }
[...]