dl9rdz / rdz_ttgo_sonde

259 stars 92 forks source link

Users should be able to see responses from SondeHub uploads. #421

Open darksidelemm opened 5 months ago

darksidelemm commented 5 months ago

If a user uploads data to SondeHub, and the data is rejected for some reason, we respond with a non-200 response code, with a description of any issues in the response.

I would recommend that if you receive any responses like this, that they are presented to the user in some manner, or be available in a log somewhere that the user can view for debugging purposes.

We have had a few situations now where users have emailed us asking why their data isn't making it onto the SondeHub map, and in many cases it's been because of DFM Z-check issues and there was no way the user could see the response we were sending back.

DG0MG commented 5 months ago

because of DFM Z-check issues

Can you explain a little bit more about this or give a link, please? Has this something to do with: https://github.com/dl9rdz/rdz_ttgo_sonde/discussions/403 ?

darksidelemm commented 5 months ago

It could be.

Information on the z-check is here: https://github.com/projecthorus/sondehub-infra/wiki/DFM-radiosonde-above-1000-and-not-enough-data-to-perform-z-check In short, we require at least 10 data packets to be uploaded in one go for DFM Sondes, due to the various issues encountered with DFM telemetry.

For auto_rx users, we recommend increasing the time between uploads (noting that all data within this period is uploaded) to 20 seconds or above if receiving DFM sondes to give a higher chance of having enough packets to pass the Z-check.

I note that rdz_ttgo_sonde has the equivalent upload time value (SONDEHUB_MAXAGE) hardcoded to 15 seconds: https://github.dev/dl9rdz/rdz_ttgo_sonde/tree/devel/RX_FSK/src

This could result in not enough packets being available for upload.

Unfortunately, without the ability to see the responses from when data is submitted, it's difficult to be sure this is exactly what's happening. However, it is likely.

hr-ru commented 5 months ago

Regarding the error code, I intend to make this info available via the About page soon.

I was not aware of this z-check. I can make the value larger, or what about this: Send a request to Sondehub if 15 packets have been received, or if more than Z (maybe 20-30 seconds) have passed since the last upload (in that case, even if it is less than 15)?

darksidelemm commented 5 months ago

The aim is to get at least 10 packets in the upload, without delaying too long. In auto_rx it ended up that increasing the time between uploads a little bit helped, so that would probably help here.

I actually do this automatically in auto_rx if receiving DFM telemetry. If DFM telemetry is seen the time between uploads is increased to 1.5x the user-set value, unless the user-set value is already greater than 30 seconds. So the default 15 second upload time ends up being ~22 seconds if DFM telemetry is being received.

hr-ru commented 5 months ago

because of DFM Z-check issues

Can you explain a little bit more about this or give a link, please? Has this something to do with: https://github.com/dl9rdz/rdz_ttgo_sonde/discussions/403 ?

This "rotating scan" feature will surely not work well with DFM. When switching to a new frequency, the receiver will wait for a significant time (this is longer for DFM as the ID is not sent with each frame) before uploading anything. If you use, e.g., rotating to a different frequency every 15 seconds, it probably will not upload any data at all (I can't tell the exact numbers from memory, but it might be something like >5 seconds for the first ID decode, and then I think its decoding the ID 1-2 more times in a row to be reasonably confident that it is correct. This means that it might take longer than 15 seconds of 100% good decoding before any data at all is sent to Sondehub (for DFM).

This is different for RS41, you have all basic information in each frame (except the calibration data for meteo), so for RS41 frames will be uploaded (without meteo) right away.