Closed eben80 closed 3 years ago
Hey,
I can't speak to the negative humidity values but I can explain the checks we have in place.
We have a few computationally light local checks: https://github.com/dl9rdz/rdz_ttgo_sonde/blob/devel/RX_FSK/RX_FSK.ino#L3271
These check for obviously wrong values like no serial number, lat/lon exactly 0, and altitude over 50,000m. The addition of a negative humidity check seems very possible.
The next step which we have recently added to the server-side includes all the checks from radiosonde_auto_rx in addition to a few extra. These can be found here: https://github.com/projecthorus/sondehub-infra/wiki/Telemetry-check-error-messages
The additional checks are the serial name regex check and a z-check to remove clear outliers from GPS data.
Thanks for the response Luke,
I didn't know there were light checks already. So far I've spotted this issue with pressure and another time when the temperature measurement was off by about 10 degrees compared to the decoded messages from an autorx station.
I mean for me it's not such a big issue and you wouldn't be able to rectify that small a temperature drift, but I just wanted to let you guys know that I "spotted it in the wild" in case something can be done like for the humidity case.
Thanks again.
Hi,
The SondeHub backend has now received support for checking against negative humidity and pressure values: https://github.com/projecthorus/sondehub-infra/commit/ece5d0e5e2df3ec85120939774e67f9061941db2
When it comes to the temperature issues I'm not so sure, it seems like something that needs to be done on-device.
Great thank you. I'm happy to close this since I don't have any data to demonstrate the temperature issue.
I can confirm that these checks are working and have rejected packets from being uploaded.
Hi. I've now been setting up a stationary device with a roof antenna that feeds telemetry into Sondehub. I've seen a couple of cases where impossible data has been sent and I wanted to know if there is a way to prevent this as its messing up the telemetry graphs even if it's just one or two frames.
Here's an example of two erroneous frames in json. Obviously humidity should never by a negative value.
` { "software_name": "radiosonde_auto_rx", "software_version": "1.5.3", "uploader_callsign": "TRGOARXMT", "uploader_antenna": "1/4 wave monopole", "time_received": "2021-09-03T00:24:20.159271Z", "datetime": "2021-09-03T00:24:36.000000Z", "manufacturer": "Vaisala", "type": "RS41", "serial": "R5030302", "subtype": "RS41-SGP", "frame": 5879, "lat": 48.74592, "lon": 20.90689, "alt": 19623.45742, "temp": -55.4, "humidity": 1.6, "pressure": 61.23, "vel_v": 4.79101, "vel_h": 5.32345, "heading": 77.19233, "sats": 8, "batt": 2.5, "frequency": 400.601, "burst_timer": 65535, "snr": 10.9, "position": "48.74592,20.90689" },
{ "software_name": "rdzTTGOsonde", "software_version": "devel20210817", "uploader_callsign": "EBSKI_RDZTTGO", "time_received": "2021-09-03T00:24:21.000Z", "manufacturer": "Vaisala", "serial": "R5030302", "datetime": "2021-09-03T00:24:38.000Z", "lat": 48.745934, "lon": 20.907043, "alt": 19632.686, "frequency": 400.6, "vel_h": 5.992, "vel_v": 5.121, "heading": 82.871, "rssi": -104, "sats": 8, "frame": 5881, "type": "RS41", "temp": -55.5, "humidity": -13314915565568, "uploader_position": "49.052277,21.282065", "uploader_antenna": "403Mhz Ground Plane", "position": "48.745934,20.907043", "uploader_alt": 272 },
{ "software_name": "rdzTTGOsonde", "software_version": "devel20210817", "uploader_callsign": "EBSKI_RDZTTGO", "time_received": "2021-09-03T00:24:22.000Z", "manufacturer": "Vaisala", "serial": "R5030302", "datetime": "2021-09-03T00:24:39.000Z", "lat": 48.745941, "lon": 20.907125, "alt": 19637.77, "frequency": 400.6, "vel_h": 6.383, "vel_v": 5.237, "heading": 81.353, "rssi": -105.5, "sats": 8, "frame": 5882, "type": "RS41", "temp": -55.5, "humidity": -12365054607360, "uploader_position": "49.052276,21.282063", "uploader_antenna": "403Mhz Ground Plane", "position": "48.745941,20.907125", "uploader_alt": 272 },
{ "software_name": "rdzTTGOsonde", "software_version": "devel20210817", "uploader_callsign": "EBSKI_RDZTTGO", "time_received": "2021-09-03T00:24:24.000Z", "manufacturer": "Vaisala", "serial": "R5030302", "datetime": "2021-09-03T00:24:40.000Z", "lat": 48.745964, "lon": 20.907303, "alt": 19646.941, "frequency": 400.6, "vel_h": 6.983, "vel_v": 3.614, "heading": 79.072, "rssi": -105.5, "sats": 8, "frame": 5884, "type": "RS41", "uploader_position": "49.052276,21.282061", "uploader_antenna": "403Mhz Ground Plane", "position": "48.745964,20.907303", "uploader_alt": 272 },`
Is there any rudimentary checks that could be done on the values before it is sent? If a number doesn't pass then send a null instead.
Thank you in advance.