Rain - something wrong with the data received

[LloydR]

This is what I found -

1. data: 03:57:08.612942 E003BE730300E26A 34901
2. data: 03:57:08.613204 E0019E310300E26A 34902
3. data: 03:57:29.113525 E001BE730300A6E9 34909 notice there are 2 ea data strings at 03:57:08 within milliseconds of each other. The 3rd byte (counting from 0 and from the left to right) is rain clicks and goes from 73 to 31 and back to 73 causing a report of 1.3" of rain. This happened twice once at 01:13;29 and 03:57:08. I expect that is probably the cause of not only rain but wind and even temperature. There should not be data coming from the GO program that quickly, neither should there be duplicate data such as DEBUG user.rtldavis: data: 12:07:33.906445 8003BC166B00A202 46260 0 0 0 0 msg.ID=0 DEBUG user.rtldavis: info: 12:07:33.908690 duplicate packet: 8003BC166B00A202 One would think the CRC would kick in but the packets have good CRC.

Further investigation warranted

11/7/22 Not only wind (I have noticed that also) but also temperature and probably any value coming into weewx-rtldavis

weewx[15687] DEBUG user.rtldavis: data: 05:00:31.352998 8004770E6A00B19E 163 0 0 0 0 msg.ID=0 weewx[15687] DEBUG user.rtldavis: data: 05:00:31.353372 800433062A00B19E 164 0 0 0 0 msg.ID=0 weewx[15687] DEBUG user.rtldavis: data: 05:00:41.601651 8002770E6B004F2A 168 0 0 0 0 msg.ID=0

11/14/22 Got an anomalous wind reading - wind was zero then all of a sudden weewx-rtldaavis reports 27 mph. Two things - once again CRC OK, Type 03 which is unknown and should have never been processed.

weewx[15687] DEBUG user.rtldavis: data_pkt: {'channel': 1, 'bat_iss': 0, 'wind_speed_ec': 0, 'wind_speed_raw': 0, 'wind_dir': 122.5494071146245, 'wind_speed': 0.0, 'temperature': -13.25, 'curr_cnt0': 111000, 'curr_cnt1': 0, 'curr_cnt2': 0, 'curr_cnt3': 0} weewx[15687] DEBUG user.rtldavis: data: 03:18:09.714547 3019B480A320EF9F 111001 0 0 0 0 msg.ID=0 weewx[15687] DEBUG user.rtldavis: data_pkt: {'channel': 1, 'bat_iss': 0, 'wind_speed_ec': 27, 'wind_speed_raw': 25, 'wind_dir': 250.9683794466403, 'wind_speed': 12.07005, 'curr_cnt0': 111001, 'curr_cnt1': 0, 'curr_cnt2': 0, 'curr_cnt3': 0}

11/29/22 Rain again - 3rd byte (starting from 0 and left) goes 5C to 1C back to 5C within milliseconds weewx[14677] DEBUG user.rtldavis: data: 07:52:59.119600 E0015B5C030034B1 51660 0 0 0 0 msg.ID=0 weewx[14677] DEBUG user.rtldavis: data: 07:53:09.368426 E0025B5C0300DA63 51664 0 0 0 0 msg.ID=0 weewx[14677] DEBUG user.rtldavis: data: 07:53:09.370324 E0021B1C02009A63 51665 0 0 0 0 msg.ID=0 weewx[14677] DEBUG user.rtldavis: data: 07:53:19.619408 E0015B5C030034B1 51669 0 0 0 0 msg.ID=0

12-21-22 Wind again - this time the type is 0 306776 weewx[14677] DEBUG user.rtldavis: data: 12:47:58.826613 00F29BE9A3EB999D 12499 0 0 0 0 msg.ID=0 306777 306778 weewx[14677] DEBUG user.rtldavis: wind_speed_raw=0f2 wind_dir_raw=0x09b 306779 weewx[14677] DEBUG user.rtldavis: WS=108.18341111111111 WD=217.17391304347825 WS_raw=242 WS_ec=242 WD_raw=155 WD_pro=217.173913 306779 04347825 WD_vue=218.26875 306780 weewx[14677] ERROR user.rtldavis: unknown message type 0x0

429318 weewx[14677] DEBUG user.rtldavis: data: 21:39:52.820657 003B264537339CA9 12201 0 0 0 0 msg.ID=0 429319 429320 weewx[14677] DEBUG user.rtldavis: wind_speed_raw=03b wind_dir_raw=0x026 429321 weewx[14677] DEBUG user.rtldavis: rx0=36, rx1=40, ry0=55, ry1=60, x0=0, x1=0, y0=0, y1=0, x=38, y=59 429322 weewx[14677] DEBUG user.rtldavis: WS=26.375294444444442 WD=59.015810276679844 WS_raw=59 WS_ec=59 WD_raw=38 WD_pro=59.0158102766 429322 79844 WD_vue=53.7375 429323 weewx[14677] ERROR user.rtldavis: unknown message type 0x0

1/31/23 I changed the golang program to reject any reading , per transmitter, that comes in sooner than 2 seconds (scan time is 2.5 seconds). Finally got 2 temperature readings, both passed CRC, one was -7.5 and the very next one was -10.7. Continuing to run with my changes in on wind and "duplicates" so I can somewhat confidently say this is a good patch until someone figures out why the SDR program does send garbage.

1. weewx[28599] DEBUG user.rtldavis: data: 18:17:19.850988 800573FB4B00EC4B 8329 0 0 0 0 msg.ID=0
2. weewx[28599] DEBUG user.rtldavis: info: 18:17:19.851412 time between reads < 2 seconds, skipping 800573F94900E449

2/4/23 weewx[28599] DEBUG user.rtldavis: data: 18:09:38.215962 A005987A29002294 4602 0 0 0 0 msg.ID=0 weewx[28599] DEBUG user.rtldavis: info: 18:09:39.886586 time between reads < 2 seconds, skipping E0EDE647FC791312

[guidocioni]

I noticed this as well :(

[gary-hammer]

For me, it's wind I've gotten wind and/or gusts of up to 125 MPH

[guidocioni]

For me, it's wind I've gotten wind and/or gusts of up to 125 MPH

Yep, I have spike outliers in both rain and winds. They are getting so frequent that it's hard to put the station reliably online :( Haven't found a way to correct them but I'm sure it's not about poor reception.

[LloydR]

At least I am not the only one. A interim fix would be to look at the time stamp. If less than say 1/2 second then throw the value away. However I have not looked at how to do that with multiple stations. There must be some way to distinguish between stations. I know Luc has the "duplicate values" set up with a previous reading and then current reading and if same == duplicate. Time stamp may be a little more difficult depending on how the multiple stations are handled.

[guidocioni]

At least I am not the only one. A interim fix would be to look at the time stamp. If less than say 1/2 second then throw the value away. However I have not looked at how to do that with multiple stations. There must be some way to distinguish between stations. I know Luc has the "duplicate values" set up with a previous reading and then current reading and if same == duplicate. Time stamp may be a little more difficult depending on how the multiple stations are handled.

Are you sure the problem with these spikes is only related to the observations being really close in time?

[LloydR]

In reply to guidocioni - Am I sure the problem is packets being close in time. I can't guarantee it but given 2 separate data points (Rain & Temp) it sure seems to me that is the problem. I am in the country, no other Davis ISS around me, I only have one Davis ISS and I am getting duplicate packages - There is something wrong with the Golang program or the RTL-SDR library. The data is supposed to be coming in every 2.5 seconds. Anything other than that is bad data in my case. By the way the duplicate packages and the data shown in the statement of the problem are from the Golang debug. Trying to get my head around what the Golang program does has proven to be difficult for me.

[guidocioni]

In reply to guidocioni - Am I sure the problem is packets being close in time. I can't guarantee it but given 2 separate data points (Rain & Temp) it sure seems to me that is the problem. I am in the country, no other Davis ISS around me, I only have one Davis ISS and I am getting duplicate packages - There is something wrong with the Golang program or the RTL-SDR library. The data is supposed to be coming in every 2.5 seconds. Anything other than that is bad data in my case. By the way the duplicate packages and the data shown in the statement of the problem are from the Golang debug. Trying to get my head around what the Golang program does has proven to be difficult for me.

Yeah I also don't know golang, I can only comment on the Python part. But, in theory, if the problem is with these milliseconds package, we could just put a sleep command or some kind of guard in the code to prevent these duplicated packages to be received.

BTW I also have a similar setup with just one station and no interference.

[gary-hammer]

I find it notable that all of us have different observations that are affected. For me, it is wind and only wind. I have limited the QC check values in WeeWX and that helps as I'm not expecting more than 85 MPH winds where I live. Still, overnight I have a 64 MPH gust recorded. Then back to the 0-3 MPH gentle breeze.

I too have eliminated all the external sources of interference and the remaining suspect is the driver.

[LloydR]

Looking at Golang code in rtldavis I have not figured out how the bad data and duplicate gets in but it is probably because of the goroutine (go func() { line 270) .

In debugging it appears that any log data that has the timestamp come from Golang rtldavis and not weewx-rtldavis. (DEBUG user.rtldavis: info: 18:00:01.149015).

Because Golang rtldavis tries to handle multiple sending devices it can not just check on current time vs previous time. It has to be indexed by transmitter number.

[LloydR]

Looks like wind is somewhat different - the weewx-rtldavis looks like it processes wind even if there is a unkown packet and the packet gets through the CRC check. That seems to be a major problem with this software suite since there are bogus packets that get through. Maybe it should check for a good packet before processing the wind data. Also noticed that the wind gust values Type 0x9 are not used; which is what is recommended since all it is is the highest value of the wind measurement for the last 9 packets.