RS41 error checking

[TheSkorm]

We should only upload if the GPS state is good and the CRC passes. This doens't seem to be happening at the moment

[rs1729]

The relevant CRC-packets are always checked for output, and in GPS3/ecef-pos altitude is checked (so ecef=(0,0,0) is not displayed).

I think not only the GPS-CRCs (GPS1, GPS3) must be ok, frame-number and frame-id, too. (frame-id would only change, if another rs41 is received)

First the CRC-check was optional, I think for rs92 it is still that way (sometimes the large rs92-GPS-block could be helpful even if CRC was incorrect.) Later the CRC-check became mandatory for each block/packet; now only the display of the crc-result for each block is optional. Reed-Solomon-ECC is still optional, but highly recommended (DFT costs probably more compute time anyway). For --json it could be turned on. --ecc2 gives the number of corrected errors, thus it indicates low signal quality.

If ECC is ok, the probability is very high that the frame is ok. On the other hand, aux-frames are nearly twice as long, so the CRC of the relevant blocks at the beginning might be enough. However once I observed a frame where the CRC for the first block (frame-nb/id) was ok, but ECC not successful. But the frame-id was the one it should have been, just the right bits were flipped such that the crc was still ok. Other blocks were invalid, otherwise the fec would kick in. https://www.fingers-welt.de/phpBB/viewtopic.php?f=14&t=43&start=2875#p245156 If for --json all the relevant CRCs must be ok, it should be very unlikely that in all the output-blocks the bits are flipped such that the CRCs are still ok, so many times that fec would not be possible.

Do you have a description of the json-structure? What about frequency and other data that is not transmitted every frame? (same for dfm09-id)

[darksidelemm]

The JSON structure is only fairly loosely defined. DECODER_REQUIRED_FIELDS = ['frame', 'id', 'datetime', 'lat', 'lon', 'alt'] DECODER_OPTIONAL_FIELDS = { 'temp' : -273.0, 'humidity' : -1, 'vel_h' : 0.0, 'vel_v' : 0.0, 'heading' : 0.0 } The optional fields are set to those values if they don't exist in the json blob.

We currently don't use the frequency data from the sonde telemetry, instead we just use the current frequency we are receiving on.

What RS41/RS92 data in particular isn't transmitted every frame?

[rs1729]

The config/calibration sub-blocks, 0x00 .. 0x32 (rs41), i.e. repeats every 51 seconds. It contains frequency, sonde-id, firmware-version, burst-kill, timers, sonde-type, ..., and calibration data for the PTU sensors. With option -v some of them are displayed. Or with -vv you get the raw-blocks.

So e.g. if the sonde-id changes, the temperature-calibration has to be loaded again.

Similar for rs92.

[darksidelemm]

OK, good to know! That would explain why temperature isn't available immediately - it has to receive the calibration blocks first. We haven't encountered any serious problems with the way the RS41 telemetry is being handled right now, so it's probably fine to be left as-is.

[TheSkorm]

@rs1729 I think the actual packets are being decoded fine without any errors however we aren't checking if the GPS has a proper fix.

Here is where we output JSON https://github.com/projecthorus/radiosonde_auto_rx/blob/master/demod/rs41dm_dft.c#L1046

When I was at the launch site the other week, I was successfully decoding the RS41, however the radiosonde was inside so didn't have a proper GPS fix and providing the incorrect location. I don't have a good way to test thing yet.

Maybe we should have requirements for pos_numSats, pos_sAcc , and pos_pDOP before uploading results online - https://github.com/projecthorus/radiosonde_auto_rx/blob/master/demod/rs41dm_dft.c#L380

What are your thoughts?

[darksidelemm]

I think an optional 'fix_status' field added to the JSON blobs (for those sondes that can provide this kind of information) would be good. At the moment the only test we do is to check that the sonde is not have a zero lat/lon, and that the altitude is in some sane range.

In the case of the RS41, I'm not sure how much of the uBlox binary format frames end up downlinked. If there is an entire NAV-SOL frame available, then we could use the gpsFix field from that (Refer: https://www.u-blox.com/sites/default/files/products/documents/u-blox6_ReceiverDescrProtSpec_%28GPS.G6-SW-10018%29_Public.pdf Section 35.9)

Otherwise, using the number of satellites is probably appropriate. I would set the threshold to >= 4 satellites.

[rs1729]

I'm afraid gpsFix is one of the few things that is not included in the 3 GPS-blocks. But numSV (Number of SVs used in Nav Solution) could be included in JSON, and maybe it is even more useful. (JSON-output only if numSV >= 4 is another choice.) (Probably it is only relevant at start up.)

Just looked through some test-recordings where I started the sonde on the ground. First the position is kind of random, then 0 0 0 and still numSV=0, then it jumps >=4 and gives the right coordinates.

I started going through the rs41-source anyway, put the radiosonde-variables in a local struct and the demod-variables, so it can be used decoding several radiosondes at the same time. And I put the configuration-data and aux-data in the rs41-struct, so it can be used for JSON-output (#88,#119). I will put the numSV to the GPS-data as well. (It is only in the --sat output.)

(#6) Then I was thinking of putting all the relevant decoding-sources together in a new repo, maybe RS_auto, so you could fork/branch from there instead of using the RS-repo that is more like a notebook for me.

Here again the blocks GPS1,GPS2,GPS3 in the rs41-frame:

[RXM-RAW 0x02 0x10] 0x093: 7C1E (L=0x1E=30 byte) 0x095: 2 byte: GPS week 0x097: 4 byte: iTOW 0x09B: 12x2 byte: SVN (1 byte) , Signal Strength/Quality (1 byte) (0x0B3: 2 byte CRC32)

0x0B5: 7D59 (L=0x59=89 byte) 0x0B7: 4 byte: minPRmes 0x0BB: 1 byte 0x0BC: 12x(4+3) byte: 4 byte PR + 3 byte DP (0x110: 2 byte CRC32)

[NAV-SOL 0x01 0x06] 0x112: 7B15 (L=0x15=21 byte) 0x114: 3x4 byte: ECEF-POS 0x120: 3x2 byte: ECEF-VEL 0x126: 1 byte: Number of SVs used in Nav Solution 0x127: 1 byte: sAcc/10 (in NAV-SOL 4 byte) 0x128: 1 byte: pDOP/10 (in NAV-SOL 2 byte) (0x129: 2 byte CRC32)

UPDATE: rs41dm_dft: numSV is now in --json output.

[TheSkorm]

Yeah, at some stage I was considering moving the demodulators to a git submodule, so we could include your stuff directly. Sounds good!

---

From: rs1729 notifications@github.com Sent: Thursday, 21 February 2019 03:13 To: projecthorus/radiosonde_auto_rx Cc: Michael Wheeler; Assign Subject: Re: [projecthorus/radiosonde_auto_rx] RS41 error checking (#111)

I'm afraid gpsFix is one of the few things that is not included in the 3 GPS-blocks. But numSV (Number of SVs used in Nav Solution) could be included in JSON, and maybe it is even more useful. (JSON-output only if numSV >= 4 is another choice.) (Probably it is only relevant at start up.)

Just looked through some test-recordings where I started the sonde on the ground. First the position is kind of random, then 0 0 0 and still numSV=0, then it jumps >=4 and gives the right coordinates.

I started going through the rs41-source anyway, put the radiosonde-variables in a local struct and the demod-variables, so it can be used decoding several radiosondes at the same time. And I put the configuration-data and aux-data in the rs41-struct, so it can be used for JSON-output (#88,#119). I will put the numSV to the GPS-data as well. (It is only in the --sat output.)

(#6) Then I was thinking of putting all the relevant decoding-sources together in a new repo, maybe RS_auto, so you could branch from there instead of using the RS-repo that is more like a notebook for me.

Here again the blocks GPS1,GPS2,GPS3 in the rs41-frame:

[RXM-RAW 0x02 0x10] 0x093: 7C1E (L=0x1E=30 byte) 0x095: 2 byte: GPS week 0x097: 4 byte: iTOW 0x09B: 12x2 byte: SVN (1 byte) , Signal Strength/Quality (1 byte) (0x0B3: 2 byte CRC32)

0x0B5: 7D59 (L=0x59=89 byte) 0x0B7: 4 byte: minPRmes 0x0BB: 1 byte 0x0BC: 12x(4+3) byte: 4 byte PR + 3 byte DP (0x110: 2 byte CRC32)

[NAV-SOL 0x01 0x06] 0x112: 7B15 (L=0x15=21 byte) 0x114: 3x4 byte: ECEF-POS 0x120: 3x2 byte: ECEF-VEL 0x126: 1 byte: Number of SVs used in Nav Solution 0x127: 1 byte: sAcc/10 (in NAV-SOL 4 byte) 0x128: 1 byte: pDOP/10 (in NAV-SOL 2 byte) (0x129: 2 byte CRC32)

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub, or mute the thread.

[rs1729]

in rs41dm_iq.c, demod_iq.c, demod_iq.h, aux-data and config-data is now included in the rs41-structure. (json is missing here.) no global variables except constants (and bch_ecc.c for now.) also demod_iq has no global variables. only ecc_bch.c I didn't change. Though I have a first working version, but a lot of changes, so I will test some more, easy to miss something. This way it could be used in a more modular way, or together in one program. So I have to adjust the other decoders in RS/demod. And when the rewritten decoders are working like the old did, I can put them together with the detect-tools in a new repository.

[darksidelemm]

Resolved by https://github.com/projecthorus/radiosonde_auto_rx/pull/138