Meteosis: help needed

[icalik]

Thanks for decoding Meteosis MTS01.

How can I run ./meteosis decoder on test branch? I have one Meteosis and when I power it it’s start to transmitting.

Can you help me for decoding it with raw recorded file or directly from rtl_fm?

Thanks

[rs1729]

The decoder expects FM audio, but keep in mind that the decoder is very simple, and it needs a clean signal with no frequency offset. The problem is that the signal has a very narrow bandwidth, so it is important to center the signal accurately. The easiest way would be to use SDR software like sdr++ (Linux/Mac/Windows) or sdr# (Windows). The IQ sample I got has a sample rate of 48000 Hz, and the signal is approx. -200 Hz off. (I believe the center frequency was 404985 kHz.) If you choose FM demodulation and a filter width of 2000-3000 Hz, you can record the FM audio or feed it directly to the decoder using sox:

sox -t pulseaudio default -t wav - 2>/dev/null | ./meteosis

If you have the raw IQ recording from rtl_fm with sample rate 48000 and assume signal offset of -200 Hz, you can use the tools csdr and sox to output FM audio for the decoder, e.g.

cat rtlfm_404985kHz_48000_iq.bin | csdr convert_i16_f | csdr shift_addition_cc `python -c "print float(200)/48000"` | csdr bandpass_fir_fft_cc `python -c "print float(-1400)/48000"` `python -c "print float(1400)/48000"` 0.01 | csdr fmdemod_quadri_cf | csdr limit_ff | sox -e float -r 48000 -b 32 -c 1 -t raw - -e signed -b 16 csdr_fm.wav lowpass 2000
./meteosis csdr_fm.wav 

or

cat rtlfm_404985kHz_48000_iq.bin | csdr convert_i16_f | csdr shift_addition_cc `python -c "print float(200)/48000"` | csdr bandpass_fir_fft_cc `python -c "print float(-1400)/48000"` `python -c "print float(1400)/48000"` 0.01 | csdr fmdemod_quadri_cf | csdr limit_ff | sox -e float -r 48000 -b 32 -c 1 -t raw - -e signed -b 16 -t wav - lowpass 2000 | ./meteosis

where python -c "print float(200)/48000" calculates the frequency to adjust for the offset, and python -c "print float(-1400)/48000" and python -c "print float(1400)/48000" are the window for the band-pass pass filter. (Or just plug in the calculated values.) (You could also use a high-pass filter to correct the frequency offset in the FM audio, 20 Hz might be enough such that the offset is corrected in the preamble.) To check if the recorded signal is promising, you can use e.g. audacityto view the FM audio sample. Should look something like this:

Perhaps you can also use rtl_fm for FM demodulation, but I'm not sure how good it is with signals that have such a small bandwidth. rtl_fm is also rather lightweight, the filtering is down via decimation, so if you choose maybe 8kHz, I would recommend to upsample to 24kHz afterwards such that the decoder has a few more samples/bit. With rtl_fm you don't "see" the signal, if you don't choose the right center frequency for the signal, you probably need at least a high pass filter to correct the offset.

There are many different ways to do it. The signal is actually very simple, but if you don't have much experience with radio signals, you have to get used to it. It is good to start with a strong signal that does not look like noise.

[icalik]

Thanks for quick response,

I just tried to decode when sonde on air, I used rtl_fm instead of cat;

rtl_fm -M raw -s 48k -f 403.0M - | csdr convert_i16_f | csdr shift_addition_ccpython -c "print float(200)/48000"| csdr bandpass_fir_fft_ccpython -c "print float(-1400)/48000"`python -c "print float(1400)/48000" 0.01 | csdr fmdemod_quadri_cf | csdr limit_ff | sox -e float -r 48000 -b 32 -c 1 -t raw - -e signed -b 16 -t wav - lowpass 2000 | ./meteosis`

but I didnt get any cout line, when I use raw recorded sound I got response from ./meteosis.

Should I change something when we want decode from rtl_fm directly?

Thanks

[rs1729]

As I said, the decoder doesn't like frequency offset. Are you sure the Meteosis signal is at 403000kHz? Or rather 402999800 Hz, since you set the frequency shift (csdr shift_addition_cc) to 200 Hz (python -c "print float(200)/48000")? (In what you posted I don't see the `-backticks around the python command, but that's probably because you quoted it in a single line, better use multi-line "add code" with 3 backticks.) Make a recording and check the offset. You could also make a FM-recording, you should see the offset, but it is more difficult to estimate the offset in the FM audio. The problem is that even a small offset could be too much because the signal itself has such a small bandwidth. You can also try using an additional high pass filter:

rtl_fm -M raw -s 48k -f 403.0M - | csdr convert_i16_f | csdr shift_addition_cc `python -c "print float(200)/48000"` | csdr bandpass_fir_fft_cc `python -c "print float(-1400)/48000"` `python -c "print float(1400)/48000"` 0.01 | csdr fmdemod_quadri_cf | csdr limit_ff | sox -e float -r 48000 -b 32 -c 1 -t raw - -e signed -b 16 -t wav - lowpass 2000 highpass 20 | ./meteosis

O try without the frequency shift and width band pass filter width -0.029 .. 0.029 where 1400/48000 = 0.029167:

rtl_fm -M raw -s 48k -f 403.0M - | csdr convert_i16_f | csdr bandpass_fir_fft_cc -0.029 0.029 0.01 | csdr fmdemod_quadri_cf | csdr limit_ff | sox -e float -r 48000 -b 32 -c 1 -t raw - -e signed -b 16 -t wav - lowpass 2000 highpass 20 | ./meteosis

If you can determine the offset, if the signal is at (403000000 - fo) Hz, use

csdr shift_addition_cc `python -c "print float(+fo)/48000"`

as frequency shift.

Unfortunately I cannot test rtl_fm with a real Meteosis signal.

UPDATE: If you don't know the exact center frequency and want to use the high pass filter, you should probably use a slightly wider IF filter, maybe 4kHz, i.e.

rtl_fm -M raw -s 48k -f 403.0M - | csdr convert_i16_f | csdr shift_addition_cc `python -c "print float(200)/48000"` | csdr bandpass_fir_fft_cc `python -c "print float(-2000)/48000"` `python -c "print float(2000)/48000"` 0.01 | csdr fmdemod_quadri_cf | csdr limit_ff | sox -e float -r 48000 -b 32 -c 1 -t raw - -e signed -b 16 -t wav - lowpass 2000 highpass 20 | ./meteosis

or

rtl_fm -M raw -s 48k -f 403.0M - | csdr convert_i16_f | csdr bandpass_fir_fft_cc -0.042 0.042 0.01 | csdr fmdemod_quadri_cf | csdr limit_ff | sox -e float -r 48000 -b 32 -c 1 -t raw - -e signed -b 16 -t wav - lowpass 2000 highpass 20 | ./meteosis

[rs1729]

Just tried with the recording, not sure if the sox high pass filter is the best choice. With 20 Hz the result is not so great. 10 Hz little better. It appears that you can choose 2 Hz or even 1 Hz, kind of works:

rtl_fm -M raw -s 48k -f 403.0M - | csdr convert_i16_f | csdr bandpass_fir_fft_cc -0.042 0.042 0.01 | csdr fmdemod_quadri_cf | csdr limit_ff | sox -e float -r 48000 -b 32 -c 1 -t raw - -e signed -b 16 -t wav - lowpass 2000 highpass 1 | ./meteosis

Seems to work up to 500 Hz offset.

[deleted the output of the recording, since the example was for rtl_fm]

[icalik]

Good news, I was able to decode it while Meteosis MTS01 radiosonde was on the air. 😊

As you said, decoder doesn't like the shifted frequency. When I searched between 400 - 406 MHz on gqrx, I saw that the sonde was 401.003MHz.

as you said, thats how I used your line:

rtl_fm -M raw -s 48k -f 401.003M - | csdr convert_i16_f | csdr bandpass_fir_fft_cc -0.042 0.042 0.01 | csdr fmdemod_quadri_cf | csdr limit_ff | sox -e float -r 48000 -b 32 -c 1 -t raw - -e signed -b 16 -t wav - lowpass 2000 highpass 1 |  ./meteosis

It was amazing to get the first console out.

Sometimes console output was pretty bad, my impression is it's probably making noise in air.

Thank you very much for your efforts. Meteosis decoder worked perfectly.

Looks like I'm going to play around with csdr parameters to get a perfect result, I guess some fuzzy logs are normal.

Now its time to hunting Meteosis radiosondes ☺️

I want to ask to would it be posible to develop demod for Meteosis now? How far are we from developing demod for Meteosis MTS01? I saw demod_base.c in RS/demod/multi/demod_base.c directory.

as a future plan, do I have any blocker to implement in the auto_rx project?

[rs1729]

gqrx is also an option, of course. If you reduce the filter bandwidth, you should be able to decode the FM audio as I explained above.

The demod/multi-folder is for decoding multiple radiosondes, but it is only for certain more common radiosondes; I don't think I will add this one. The demod/mod-folder contains radiosondes where the decoder uses the demodulation functions from demod_mod.c. It can also decode IQ data. Maybe I do a similar version for meteosis, perhaps it will be able to handle frequency offset. This long "zero"-line at the end of each frame is kind of disturbing, though.

For now, also try the option -b in the meteosis decoder. For -b the FM decoding is in principle the same as for the *mod-decoders, only the header/sync-detection is weaker.

rtl_fm [...] |  ./meteosis -b

[icalik]

No, it didnt worked. I tried with Meteosis sonde,

rtl_fm -M raw -s 48k -f 404.984M - |  sox -e float -r 48000 -b 32 -c 1 -t raw - -e signed -b 16 -t wav - lowpass 2000 highpass 1 | ./meteosis -b

and, tried. recorded audio;

cat rtlfm_mts01_20220829_1_12Z_404985Hz_48000_iq.bin | ./meteosis -b 

and tried without sox.

While trying to decode from the air for the last 3 4 days, I got very good results by manually shifting the frequency.

Is there another radiosonde with FSK modulation? If there is, it will inspire me to prepare a demod for Meteosis.

[rs1729]

The meteosis decoder cannot read IQ data. You have to convert to FM and you need a wav header.

cat rtlfm_mts01_20220829_1_12Z_404985Hz_48000_iq.bin | csdr convert_i16_f | csdr bandpass_fir_fft_cc `python -c "print float(-2000)/48000"` `python -c "print float(2000)/48000"` 0.01 | csdr fmdemod_quadri_cf | csdr limit_ff | sox -e float -r 48000 -b 32 -c 1 -t raw - -e signed -b 16 -t wav - lowpass 2000 highpass 1 | ./meteosis -b

I put together meteosis_mod.c that can read IQ data (raw or wav) and also corrects frequency offset. I didn't test much, have only one sample file. You need the files demod_mod.c and demod_mod.h from RS/demod/mod, then compile:

    gcc -O3 -c demod_mod.c
    gcc meteosis_mod.c demod_mod.o -lm -o meteosis_mod

Now you can do ./meteosis_mod --IQ <fq> [--dc] <iq_data.wav> where <fq> is the frequency offset (the negative shift value of csdr), --dc is option for frequency offset correction, though without --dcyou have to set the offset pretty accurate. Examples:

./meteosis_mod --IQ -0.004 --dc rtlfm_mts01_20220829_1_12Z_404985Hz_48000_iq.wav

or decode raw data from rtl_fm

./meteosis_mod --IQ -0.004 --dc - 48000 16 rtlfm_mts01_20220829_1_12Z_404985Hz_48000_iq.bin

if you have a raw IQ file without wav header, - 48000 16 means sample rate 48000 and 16 bit per sample. If the frequency offset is bigger, it can take a few frames until it adjusts. E.g. try offset of +5kHz:

./meteosis_mod --IQ 0.1 --dc - 48000 16 rtlfm_mts01_20220829_1_12Z_404985Hz_48000_iq.bin

Didn't test the low pass filters that could help decoding weak signals. I guess with --lpIQ the inital offset should not be more than perhaps 3kHz.

BTW Yes, almost all radiosondes use FSK, a few AFSK.

[darksidelemm]

Just a question on this sonde - is it actually sending that CSV data over the air? Is there a CRC?

[icalik]

@darksidelemm yes, sth like that, no CRC.

2 days ago I realized 5th sector is probably battery voltage, I'm open to discussion to find other fields

SONDEID,?,FRAME,YYMMDDHHMMSS,?,LAT,LON,ALT,?,?........??

10008528,2,361,220829215823,4800,40.923134,29.302095,127,0.0,0.1,7,5.31,5.31,237.35,4,2,1,0
10008528,2,362,220829215825,4800,40.923130,29.302095,127,0.0,0.0,7,5.30,5.30,237.50,4,2,1,0
10008528,2,363,220829215827,4800,40.923134,29.302099,127,0.0,0.1,7,5.32,5.32,237.29,4,2,1,0
10008528,2,364,220829215829,4800,40.923138,29.302099,127,0.0,0.1,7,5.31,5.31,237.25,4,2,1,0
10008528,2,365,220829215831,4800,40.923138,29.302099,127,0.0,0.1,7,5.31,5.31,237.40,4,2,1,0
10008528,2,366,220829215833,4800,40.923141,29.302099,128,0.0,0.1,7,5.31,5.31,237.23,4,2,1,0
10008528,2,367,220829215835,4800,40.923138,29.302099,128,0.0,0.0,7,5.31,5.31,237.30,4,2,1,0
10008528,2,368,220829215837,4800,40.923134,29.302099,128,0.0,0.2,8,5.32,5.32,237.18,5,2,1,0
10008528,2,369,220829215839,4800,40.923126,29.302095,129,0.0,0.0,7,5.33,5.33,237.40,4,2,1,0
10008528,2,370,220829215841,4800,40.923126,29.302095,129,0.0,0.1,7,5.31,5.31,237.51,4,2,1,0

[darksidelemm]

If there's no CRC, then I do not want telemetry from these sondes uploaded to SondeHub, sorry.

It will be FAR too easy for corrupt telemetry to result in problems on the maps. We've already had these kinds of issues happen in the past from DFM and RS92 sondes (not for entirely the same reasons, but with similar results).

[darksidelemm]

You can already see the issue presenting in this screenshot, where we have position corruption issues, and multiple callsigns appearing.. Not a fan!

[darksidelemm]

I should clarify here - I think adding decoding support in auto_rx is probably fine, however I don't want it uploaded to Sondehub.

[rs1729]

Yes, the raw data bytes is the csv. Then there are a lot of 00 bytes and then there might be 2 random looking bytes before the transmission stops. Maybe it's a checksum and room for additional data, don't know, didn't investigate much further, only have a few frames of raw data. On the other hand it would be strange if there would not be a checksum or crc.

[darksidelemm]

Hmm, perhaps @icalik can provide some IQ recordings...

[icalik]

@darksidelemm OK OK. multiple call signs should not be as you say and no one wants that. I understand why you don't want it to be uploaded to Sondehub, when I look at auto_rx history, wrong call sign is 1 time every time. I can keep sonde_id of the previous 3 heartbeat in the decoder, compare new heartbeat with this one, and ignore it if it's different. Thus, different sonde_IDs do not appear on the sondehub during the flight.

@rs1729 @darksidelemm I will record IQ records today.

[rs1729]

The ID is most important, but also time and position should be reliable. If you are the only one uploading, that might be a minor problem, but still it would be good to have unique frames.

If you have a Meteosis at home, maybe you can also make recordings of startup sequence. GPS lock is not important, and we don't need your home location. Maybe you make the recording somewhere outside.

[rs1729]

Took another look at the data, the 16 bits at the end are indeed the CRC-16 value of the data. Should be over the 128 data bytes including the 30 or more zero bytes at the and, probably space for other optional data. Updated the sources, though I did have only this one recording with a few frames for testing. @icalik If you can make some more recordings would be nice. No need for startup recording, normal recording of in-flight/sounding should be as good.

So in principle it could be used for uploading to sondehub, ID and CRC is available, also a frame counter. (Would be interesting to see a count above 999.) I can add json-output (though I think you already did).

@darksidelemm Don't know if the Meteosis radiosonde has to be added to the search/detect of auto_rx, maybe it is sufficient if it is prepared for manual upload? (Is UDP upload still possible?)

[rs1729]

added basic json output. sonde type as in the image above: "MTS01". @icalik not sure, where the "S" in front of the serial comes from!? Don't know, if time is UTC or GPS, and if altitude is MSL or GPS/ellipsoid.

10008528,2,362,220829215825,4800,40.923130,29.302095,127,0.0,0.0,7,5.30,5.30,237.50,4,2,1,0  [OK]
 [ 362]  (10008528)  2022-08-29 21:58:25  lat: 40.923130  lon: 29.302095  alt: 127 
{ "type": "MTS01", "frame": 362, "id": "MTS01-10008528", "datetime": "2022-08-29T21:58:25.000Z", "lat": 40.92313, "lon": 29.30210, "alt": 127.00000, "freq": 404985 }

[icalik]

Hi I just make new record with this pipe:

rtl_fm -M raw -F9 -T -p 0 -d 1 -s 48000 -f 401000000 - > mts01_10112022_48k_401MHz.bin

you can find here: https://mega.nz/folder/Xk8mxTbA#Fb6sG416Y4LzpM5_qRxkIA

@rs1729 I guess time is UTC. I also added METEOSIS to dft_detect. Can you check my last 5 commits on the testing branch in my forked repository? (https://github.com/icalik/radiosonde_auto_rx/commits/testing)

[icalik]

I realized it, but 5th section (4800) looks like battery voltage. I'm thinking of increasing it up to 5.1 volts with power supply just to be sure. (before lat value)

[darksidelemm]

Adding it to dft_detect would be useful, as then the auto_rx scan->decode system will work, but we need to be careful that it doesn't result in false detections with any other sondes.

[rs1729]

I can check the detect_dft suggestion. There is always the option to make it optional via #ifdef, only for users that have these radiosondes in their neighborhood. At least for testing. UPDATE: @icalik I don't see any changes to scan/dft_detect.c. I see that you choose other file names, also the serial number/ID with "S...", is the "S" on the envelope? It is not in the data, right?

I can look at detection and dft_detect myself this week-end.

[darksidelemm]

From the one photo I have seen of the outside of a MTS01, it looks like it doesn't have any prefix to the number:

[darksidelemm]

Further to that, with the serial number being 8 digits (same as what your decoder produces for the DFM-17's, noting that the serial number on the DFM17 box is actually of the form AAAAA-BBBBBB), there is a possibility of serial number overlap with DFM sondes.

[rs1729]

As in the example above, I would prefix "MTS01-" to the serial number:

10008528,2,362,220829215825,4800,40.923130,29.302095,127,0.0,0.0,7,5.30,5.30,237.50,4,2,1,0  [OK]
 [ 362]  (10008528)  2022-08-29 21:58:25  lat: 40.923130  lon: 29.302095  alt: 127 
{ "type": "MTS01", "frame": 362, "id": "MTS01-10008528", "datetime": "2022-08-29T21:58:25.000Z", "lat": 40.92313, "lon": 29.30210, "alt": 127.00000, "freq": 404985 }

[darksidelemm]

Yep, I can work with that :-)

Some of the other fields look like they might be usable? After the altitude, i'm guessing the next two might be vertical and horizontal velocity, then number of sats? Not sure about the rest.

[rs1729]

@icalik The recordings show altitude 130m on 2022-08-29 and 125m on 2022-11-10. According to "FreeMapTools" the actual elevation at this location is 112m, although it might be that the recording was made on top of the building?! Geoid altitude is 37m. The reported altitude is somewhere in between...

If time is UTC or GPS, this could be easily determined if you decode live data. GPS time would be 18 sec ahead.

@darksidelemm for velocity it would be good to have data from ascend and descend.

[darksidelemm]

OK, so I think next steps would be to get MTS01 decoding support into auto_rx testing as an initial step, but also support the saving out of the raw data (like is supported for many other sonde types already). That way some complete flights of the raw data can be captured for further analysis.

[darksidelemm]

I've added MTS01 decode support to the auto_rx testing branch (-beta14) in a preliminary fashion (no scan support yet, will wait on @rs1729 implementing that). Ideally I need one of these things to test with, but I should be able to make do with the IQ data for a while.

To start up auto_rx decoding a MTS01, you'd need to use: python3 auto_rx.py -m MTS01 -f 401.000 -v

If you enable the 'save_raw_hex' output option (here: https://github.com/projecthorus/radiosonde_auto_rx/blob/testing/auto_rx/station.cfg.example#L465) then any non-json output from the decoder will be saved to the log directory.

Note that when uploading data to SondeHub I'm using the serial number 'as-is', no characters prefixed. I've also decided that for APRS uploads, I'll convert the serial number to hexadecimal, and create an APRS Object ID of the form: MTSxxxxxx, where the x's are the last 6 hex nibbles of the serial number.

[rs1729]

Later today I want to look at dft_detect.

Probably I asked this before, if you send the bare serial number that might overlap with DFM numbers, are these uploads separated by the sonde type? If so, if I search for a serial number that is assigned to two different radiosonde, would it find both? Maybe it will not be a problem here anyway, the DFM serial seems start with the YY, MTS01 is now at 10. But 8-digit serials are quite common. EDIT: Did you use the decoder https://github.com/rs1729/RS/blob/test/meteosis/meteosis_mod.c? Probably I will rename and move it to the mod-directory, and copy to master.

[icalik]

Yes @rs1729 this is reason why I put S the before SondeID. On the recording I have made it on my balcony and yes I'm on 4th floor. I can re-record when sonde is on the ground.

btw I have tried to implement Meteosis to dft_detect. just added header and increase Nrs.

HERE

[icalik]

@darksidelemm I can provide more pics of Meteosis MTS01.

[darksidelemm]

@rs1729 yep i used that one. i'll replace it with whatever ends up in demod/mod - I just wanted to get something in there.

At the moment Sondehub only searches by serial, but we will likely need to add selection by type if this becomes a problem. I think you are right and there shouldn't be overlap though, at least based on what we see so far.

@icalik I've already got good pictures of the inside of one. What would be useful is a long (2 minute) IQ recording, just like you made previously. I can then use this to check demodulator performance.

[rs1729]

@icalik so the reported altitude is rather the altitude above MSL, I guess. And your observations are that time is UTC?

Do you have a recording of a sounding of the weather station in your area? Perhaps a minute on ascent and another on descent? Probably the velocity data becomes obvious.

My priority would be 1) time & date, position 2) serial number, frame counter, crc/checksum 3) velocity 4) PTU 5) other data

[rs1729]

renamed meteosis_mod.c -> mts01mod (didn't move to mod-folder yet); added MTS01 to dft_detect.c The type is reported as "MTS01", as is the "type" in the JSON output.

I added a third IF/IQ filter bandwidth, because the MTS01 bandwidth is so small. Because of the low baud rate the header has a lot of samples. Testing showed an increase of approx 20% processing time. It could be reduced if shorter header is chosen, but then it would not be as accurate I guess. More radiosonde types means more detection work. One possibility to reduce the work load could be to make certain radiosondes optional, e.g. C34/C50, iMet1-AB, MTS01.

[darksidelemm]

Currently i dont even support decoding for these three in auto_rx:

    { 9600, 0, 0, imet1ab_header, 1.0, 0.0, 0.80, 2, NULL, "IMET1AB",  tn_IMET1ab, 1, 3, 0.0, 0.0}, // (rs_hdr[idxAB])
    { 9600, 0, 0, imet_preamble,  0.5, 0.0, 0.80, 4, NULL, "IMETafsk", tn_IMETa  , 1, 1, 0.0, 0.0}, // IMET1AB, IMET1RS (IQ)IMET4
    { 9600, 0, 0, imet1rs_header, 0.5, 0.0, 0.80, 2, NULL, "IMET1RS",  tn_IMET1rs, 0, 3, 0.0, 0.0}, // (rs_hdr[idxRS]) IMET4: lpIQ=0 ...

They are also a lot less common now. So dropping those would be a way to drop some CPU I guess.

[darksidelemm]

I've run the latest dft_detect over my sample set with and without the NOMTS01 option (I always had NOC34C50 included). Results are below. 'error' indicates no detection.

This was all with 20 kHz detection bandwidth. As a result the MTS01 requires a higher SNR to be detected, which is to be expected I guess.

Definitely an observable increase in processing time, but I was running this on a M1 Macbook. Should test this on something like a RPi. I suspect the extra processing time wont really be that much of an issue. I would be interested to see how much it drops by removing those three iMet sonde types.

dft_detect_noc34_nomts01.txt dft_detect_noc34.txt

[rs1729]

I rearranged the ifdef-options, but did not test much, more tomorrow. Now you can exclude NOMTS01, NOC34C50 and/or NOIMET1AB. If there are no such radiosondes, it reduces the computations. However if there is a MTS01, C50 or IMET1AB signal, it will try to detect until end of file or time limit.

Perhaps the lowest IQ filter bandwidth could be reduced to 4kHz, this would probably detect more MTS01. Now it is at 6kHz. But I don't have many recordings for testing MTS01. If MTS01 is to be used, the header can be shortened (one of the "AA"), this reduces work load as well.

[darksidelemm]

Do you mean there are multiple filters in use in dft_detect? I'm just setting --bw to 20 when searching for 400 MHz sondes.

I had a go doing some runs with the different compile-time options.

With a 5 second long input (setting -t 5 ), just working on noise (worst-case) I end up with:

(no defines) = ~0.210 seconds NOC34C50 = ~0.196 seconds NOC34C50 + NOMTS01 (effectively what is currently in use in auto_rx) = ~0.170 seconds NOC34C50 + NOIMET1AB = ~0.182 seconds

(all on my Macbook)

So, I think the tradeoff of dropping detection support for the IMET1AB telemetry is probably OK. We don't really see them much (at all?) anymore.

[rs1729]

For IQ detection it had 2 filter width, 12kHz and 22kHz; for MTS01 I added also 6kHz. It produces now 4 FM buffers in parallel, for each filter bandwidth and 1 without filtering. Then each sonde type chooses the FM stream that it is set to, i.e. RS41 has 12kHz, M20 has 22kHz, DFM has 12kHz etc. The new MTS01 would use 6kHz. (So there is now an additional buffer just for MTS01.)

Now if you set --bw 20 you overwrite the lower 3 lpIQ_bw with 20kHz, i.e. all radiosondes would be detected with 20kHz lowpass filter (except IMET1AB, MK2LMS, IMET1RS which have wider bandwidth, so no filtering).

The --bw option is primarily for setting the IF bandwidth to a higher value than 48kHz for radiosondes that have more than 48kHz bandwidth such as IMET1RS and MK2LMS. In 400MHz only the IMET1RS would need something like --bw 64, and in L band the MK2a/LMS6 (maybe --bw 200?).

If you use input bandwidth of 48kHz IQ, test MTS01 without --bw, should be much better. Also the standard radiosondes width bandwidth in the range 8-12 kHz would be detected weaker if you set --bw 20.

The IQ lowpass filtering is more important in dft_detect because it is doing FM demodulation before scanning for the individual headers. With FM it does not need to no the modulation index for each radiosonde type, it can prepare only 3 or 4 basic buffer instead of demodulating for each of the 12+ radiosonde types.

If you don't try to detect the iMet-1-RS width 60kHz bandwidth, better do not use the --bw option in the 400MHz band. For iMet-1-RS you would need more than 48kHz IQ anyway, right? So it is probably not the same receiver output as for the other radiosondes.

I have to look, maybe I try to restrict to 1 FM stream (only 1 filter) if --bw is set. Another possibility would just to overwrite one of the filter setting or the last one that does not filter.

[rs1729]

The iMet-1-AB that I mean is the one that was used in Belgium several years ago. Now they switched to iMet-4. The iMet-1-AB had 32kHz bandwidth and a different format than iMet-1-RS/iMet-4. It was more 1200/2400 and kind of Trimble frames, although they switched to ublox at the end. The box looked the same as iMet-1-RS, old fashioned and heavy.

[darksidelemm]

Significant improvement without the --bw option! (Except for the RS92-NGP, as expected)

dft_detect_nobw.txt

However, the downside is that if there is any frequency shift to the sonde (e.g. off by 5 kHz from the centre frequency), then detection doesn't occur. This will always be the tradeoff i guess...

I'm also thinking its best to set the detection threshold to around about the same point (or a little lower) at which telemetry can be decoded. With no --bw option i think the detection point is easily 5-8 dB lower than the point at which telemetry can be decoded. This might cause some confusion with users who would be wondering why they can detect and not decode.

[rs1729]

For RS92-NGP there was the L band option which sets the lpIQ presets higher, if I remember correctly. RS92 would use 32kHz filter width with option -L. Hm... MK2LMS would use 400kHz (i.e. no filtering), though there is also 200kHz in between. Also I'm not sure right now, if the IF target bandwidth is adjusted for L band if using --IQ .... Maybe I have to add it, then -L would be enough to cover RS92-NGP and MK2a/LMS6-1680. (The only real problem is iMet-1-RS in 400MHz.) How much L band is there in use, is the transition completed?

Well, dft_detect was growing over time... it's a bit overloaded and messy.

Shift is only a problem if the peak detection is off. The filters are already chosen a bit wider such that the signal is not cut off. But there is a trade off. Wide bandwidth signals are easier to handle, however the MTS01 with its narrow bandwidth is a bit more problematic. The --dc really helps the MTS01 detection, and even more important for the MTS01 decoder.

[rs1729]

Lowering the threshold could potentially increase the possibility of false detection I believe. (Another trade off consideration)

BTW Tested my old FM collection, 3716 detections, no MTS01, i.e. no new false detection here.

[rs1729]

Another parameter outside dft_detectwould be the scan/detect frequency. On a RPi this could make a difference. On the one hand you want the detection as soon as the signal pops up, but then the scanner is running all the time...

One could also think of two detectors, one for very common radiosondes that runs more often, and one for less common radiosondes or all known types, that runs only once per minute.

[darksidelemm]

With the network-sdr option (currently using spyserver), the scanning loop does end up running continuously, as there's always enough receive channels available. On systems with just one RTLSDR thats a bit harder, as we've discussed previously.

Anyway, I've added this into the testing branch, though still with the --bw 20 option (which does seem to be performing OK 'in the wild'). I use --bw 20 when detecting in 400 MHz, but for 1680 MHz its set wider. Only a few holdout stations running 1680 MHz sondes now, they should all be cut over very soon.

[rs1729]

If you want to use the fixed 20kHz bandwidth, I could reduce the number of lowpass operations, only one is needed then instead of the three for 6, 12 and 22 kHz signals. But you might test without --bw 20, should detect better if peak detection is not off too much.

[darksidelemm]

Yeah part of the problem is i do some frequency quantization in the scan / peak-detection step, to avoid heap of clustered peaks showing up. Currently I quantise to 10 kHz steps, though this is adjustable. This does mean that in the absolute worst case a sonde could be +/- 5 kHz off the centre frequency used in the detection. I think the majority of times this isn't the case though.

[rs1729]

Just updated dft_detect such that setting --bw will do only one IQ low pass. Did not do performance test with IQ data, and for FM input it is no difference, no IQ filtering there. Maybe you can test IQ, you can run a loop of several rounds.

With the -L option we have the problem that RS92-NGP is below 48kHz (which is the default IF bandwidth for down sampling) and MK2a/LMS6-1680 with 180kHz is way above, so you have to set the higher input bandwidth anyway. Now I changed the Mk2a-filter to 22kHz, which becomes 200kHz if used with -L. Before it was no filter. Don't know which version is better, hm... Would only make a difference if you use --iq instead of --IQ 0.0 I believe. If you set --bw 180 or --bw 200 it should not matter.

On could think of producing two streams, one down sampled to 48kHz, one at 200kHz, but not now. Or maybe never, because it is becoming obsolete soon.