Wu-client not mapping data

[matbellg]

Hi,

I configured Weewx and Interceptor with success. Configuration in the weew.conf:

driver = user.interceptor
device_type = wu-client
promiscuous = 0
mode = listen
port = 8081

In the network trace I see the WU Client data passed from the device:

23:14:39.221042 IP (tos 0x0, ttl 128, id 42259, offset 0, flags [DF], proto TCP (6), length 480) 192.168.50.107.14751 > 192.168.50.6.8081: Flags [P.], cksum 0x5746 (correct), seq 1:441, ack 1, win 5840, length 440 0x0000: 4500 01e0 a513 4000 8006 6e42 c0a8 326b E.....@...nB..2k 0x0010: c0a8 3206 399f 1f91 1aea 66b2 3c47 34bc ..2.9.....f.<G4. 0x0020: 5018 16d0 5746 0000 4745 5420 3149 443d P...WF..GET.1ID= 0x0030: 3126 5041 5353 574f 5244 3d31 2669 6e64 1&PASSWORD=1&ind 0x0040: 6f6f 7274 656d 7066 3d37 322e 3326 7465 oortempf=72.3&te 0x0050: 6d70 663d 3531 2e33 2664 6577 7074 663d mpf=51.3&dewptf= 0x0060: 3530 2e32 2677 696e 6463 6869 6c6c 663d 50.2&windchillf= 0x0070: 3531 2e33 2669 6e64 6f6f 7268 756d 6964 51.3&indoorhumid 0x0080: 6974 793d 3533 2668 756d 6964 6974 793d ity=53&humidity= 0x0090: 3936 2677 696e 6473 7065 6564 6d70 683d 96&windspeedmph= 0x00a0: 302e 3026 7769 6e64 6775 7374 6d70 683d 0.0&windgustmph= 0x00b0: 302e 3026 7769 6e64 6469 723d 3431 2661 0.0&winddir=41&a 0x00c0: 6273 6261 726f 6d69 6e3d 3239 2e35 3630 bsbaromin=29.560 0x00d0: 2662 6172 6f6d 696e 3d32 392e 3632 3826 &baromin=29.628& 0x00e0: 7261 696e 696e 3d30 2e30 3030 2664 6169 rainin=0.000&dai 0x00f0: 6c79 7261 696e 696e 3d30 2e36 3330 2677 lyrainin=0.630&w 0x0100: 6565 6b6c 7972 6169 6e69 6e3d 302e 3730 eeklyrainin=0.70 0x0110: 3126 6d6f 6e74 686c 7972 6169 6e69 6e3d 1&monthlyrainin= 0x0120: 312e 3938 3026 736f 6c61 7272 6164 6961 1.980&solarradia 0x0130: 7469 6f6e 3d30 2e30 3026 5556 3d30 2664 tion=0.00&UV=0&d 0x0140: 6174 6575 7463 3d32 3032 302d 3130 2d31 ateutc=2020-10-1 0x0150: 3525 3230 3231 3a31 343a 3337 2673 6f66 5%2021:14:37&sof 0x0160: 7477 6172 6574 7970 653d 4561 7379 5765 twaretype=EasyWe 0x0170: 6174 6865 7256 312e 352e 3426 6163 7469 atherV1.5.4&acti 0x0180: 6f6e 3d75 7064 6174 6572 6177 2672 6561 on=updateraw&rea 0x0190: 6c74 696d 653d 3126 7274 6672 6571 3d35 ltime=1&rtfreq=5 0x01a0: 2048 5454 502f 312e 300d 0a48 6f73 743a .HTTP/1.0..Host: 0x01b0: 2031 3932 2e31 3638 2e35 302e 360d 0a41 .192.168.50.6..A 0x01c0: 6363 6570 743a 2a2f 2a0d 0a43 6f6e 6e65 ccept:/..Conne 0x01d0: 6374 696f 6e3a 2043 6c6f 7365 0d0a 0d0a ction:.Close....

But in the weewx web page I don't see any data. The debug log shows:

Oct 15 21:42:55 WEEWXPC weewx[3032] DEBUG user.interceptor: raw data: Oct 15 21:42:55 WEEWXPC weewx[3032] DEBUG user.interceptor: raw packet: {'dateTime': 1602790975, 'usUnits': 1} Oct 15 21:42:55 WEEWXPC weewx[3032] DEBUG user.interceptor: mapped packet: {'dateTime': 1602790975, 'usUnits': 1}

it seems a problem with the parser. Could you please give me a feedback to fix this issue?

thanks,

[KC1ELF]

This may be totally unrelated, but I will point it out just in case.

Weather Underground had an issue on 10/13 and was not populating data on its website. It seems to be back up and running again as of 10/14.

Best Regards,

Tom Kavanaugh

From: matbellg [mailto:notifications@github.com] Sent: Thursday, October 15, 2020 5:20 PM To: matthewwall/weewx-interceptor weewx-interceptor@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: [matthewwall/weewx-interceptor] Wu-client not mapping data (#85)

Hi,

I configured Weewx and Interceptor with success. Configuration in the weew.conf:

driver = user.interceptor device_type = wu-client promiscuous = 0 mode = listen port = 8081

In the network trace I see the WU Client data passed from the device:

23:14:39.221042 IP (tos 0x0, ttl 128, id 42259, offset 0, flags [DF], proto TCP (6), length 480) 192.168.50.107.14751 > 192.168.50.6.8081: Flags [P.], cksum 0x5746 (correct), seq 1:441, ack 1, win 5840, length 440 0x0000: 4500 01e0 a513 4000 8006 6e42 c0a8 326b E.....@...nB..2k mailto:E.....@...nB..2k 0x0010: c0a8 3206 399f 1f91 1aea 66b2 3c47 34bc ..2.9.....f.<G4. 0x0020: 5018 16d0 5746 0000 4745 5420 3149 443d P...WF..GET.1ID= 0x0030: 3126 5041 5353 574f 5244 3d31 2669 6e64 1&PASSWORD=1&ind 0x0040: 6f6f 7274 656d 7066 3d37 322e 3326 7465 oortempf=72.3&te 0x0050: 6d70 663d 3531 2e33 2664 6577 7074 663d mpf=51.3&dewptf= 0x0060: 3530 2e32 2677 696e 6463 6869 6c6c 663d 50.2&windchillf= 0x0070: 3531 2e33 2669 6e64 6f6f 7268 756d 6964 51.3&indoorhumid 0x0080: 6974 793d 3533 2668 756d 6964 6974 793d ity=53&humidity= 0x0090: 3936 2677 696e 6473 7065 6564 6d70 683d 96&windspeedmph= 0x00a0: 302e 3026 7769 6e64 6775 7374 6d70 683d 0.0&windgustmph= 0x00b0: 302e 3026 7769 6e64 6469 723d 3431 2661 0.0&winddir=41&a 0x00c0: 6273 6261 726f 6d69 6e3d 3239 2e35 3630 bsbaromin=29.560 0x00d0: 2662 6172 6f6d 696e 3d32 392e 3632 3826 &baromin=29.628& 0x00e0: 7261 696e 696e 3d30 2e30 3030 2664 6169 rainin=0.000&dai 0x00f0: 6c79 7261 696e 696e 3d30 2e36 3330 2677 lyrainin=0.630&w 0x0100: 6565 6b6c 7972 6169 6e69 6e3d 302e 3730 eeklyrainin=0.70 0x0110: 3126 6d6f 6e74 686c 7972 6169 6e69 6e3d 1&monthlyrainin= 0x0120: 312e 3938 3026 736f 6c61 7272 6164 6961 1.980&solarradia 0x0130: 7469 6f6e 3d30 2e30 3026 5556 3d30 2664 tion=0.00&UV=0&d 0x0140: 6174 6575 7463 3d32 3032 302d 3130 2d31 ateutc=2020-10-1 0x0150: 3525 3230 3231 3a31 343a 3337 2673 6f66 5%2021:14:37&sof 0x0160: 7477 6172 6574 7970 653d 4561 7379 5765 twaretype=EasyWe 0x0170: 6174 6865 7256 312e 352e 3426 6163 7469 atherV1.5.4&acti 0x0180: 6f6e 3d75 7064 6174 6572 6177 2672 6561 on=updateraw&rea 0x0190: 6c74 696d 653d 3126 7274 6672 6571 3d35 ltime=1&rtfreq=5 0x01a0: 2048 5454 502f 312e 300d 0a48 6f73 743a .HTTP/1.0..Host: 0x01b0: 2031 3932 2e31 3638 2e35 302e 360d 0a41 .192.168.50.6..A 0x01c0: 6363 6570 743a 2a2f 2a0d 0a43 6f6e 6e65 ccept:/..Conne 0x01d0: 6374 696f 6e3a 2043 6c6f 7365 0d0a 0d0a ction:.Close....

But in the weewx web page I don't see any data. The debug log shows:

Oct 15 21:42:55 WEEWXPC weewx[3032] DEBUG user.interceptor: raw data: Oct 15 21:42:55 WEEWXPC weewx[3032] DEBUG user.interceptor: raw packet: {'dateTime': 1602790975, 'usUnits': 1} Oct 15 21:42:55 WEEWXPC weewx[3032] DEBUG user.interceptor: mapped packet: {'dateTime': 1602790975, 'usUnits': 1}

it seems a problem with the parser. Could you please give me a feedback to fix this issue?

thanks,

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/matthewwall/weewx-interceptor/issues/85 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AE7CISLP7B7RSFMA3E7FHSDSK5RQFANCNFSM4SSQPF5Q . https://github.com/notifications/beacon/AE7CISMDWV6XMW3HQGJT5O3SK5RQFA5CNFSM4SSQPF52YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4KYTI6MA.gif

[matbellg]

Thanks Tom, I've seen the problem with the upload on Weather Underground but here the problem is local network from the WS and the Linux PC with Weewx installed.

Hope someone would help me.

[AlexTrl]

Hi, I am having exactly the same issue. I will let you know if I make any progress.

Cheers

Alex

[Alvipas]

The solution given in issue #82 works for me. Hopefully someone with more knowledge on the code can start a pull request

[jonkloske]

I updated my EasyWeather clone wifi weather station to firmware version 1.5.7 and this started happening to me. It seems the issue is that when it's talking to the genuine Weather Underground reporting service, it issues the following GET request:

GET /weatherstation/updateweatherstation.php?ID=... etc

However, when using the custom reporting service with the "wunderground compatible" mode (I can't remember the precise wording, and I'm not on the station wifi at the moment) it seems to drop the path entirely, so the GET request looks like this:

GET ID=... etc

This is obviously a bug in the protocol, but I haven't looked hard enough at the app settings (I'll do so later this evening if I get a chance) to see if there's a way to rescue it from the settings side of things.

The way the weewx-interceptor works when in "listen" mode is it parses the GET request and pulls out the QUERY portion and then gets to work on it. Sadly, the buggy firmware is not supplying a path or even a ? at the start of the query string, so the entire thing gets interpreted as the PATH and the QUERY portion is empty:

def do_GET(self):

get the query string from an HTTP GET

data = urlparse.urlparse(self.path).query

So, there's at least five possible fixes here:

1. The EasyWeather firmware peeps update their firmware to fix the issue (or we figure a workaround based on the in-app settings).

2. Switch to using the "sniff" mode (if that's an option for you) - since the actual wunderground GET request works fine

3. Modify the "data = " line above locally to retrieve the .path instead of the .query - just be aware this will break any other weather stations that are working correctly. Eg:

   data = urlparse.urlparse(self.path).path

4. Modify the "data = " line to include a fake query part, by adding a '?' to the start of the self.path - again, this will probably break other weather stations that are working correctly. Eg:

   data = urlparse.urlparse('?' + self.path).query

5. There's a 5th option in a linked post above, where the poster isn't sure why it works but it does, where they just skip the parsing entirely (since it's effectively pre-parsed_. and just set the data directly to the self.path, eg:

   data = self.path

Again, that will break your install for non-broken weather stations.

Assuming this problem persists for a longer time and we want to actually support this broken behaviour in weewx-interceptor, one option would be to scan the self.path string initially for the broken software version, and modify our parsing behaviour based on that. Or, we could make it a config option (basically a WUClient processor version that is configured to expect the buggy behaviour). It would certainly make the code a bit uglier, though, so I understand why it might not be the first option. On my system, the broken version string is at least:

softwaretype=EasyWeatherV1.5.7

Based on people reporting this earlier than 2021, it's possible multiple versions and software is affected though, so the config option certainly seems like a broader solution than trying to maintain a list of buggy versions internally (the only real benefit of doing that is it would "just work" for most people who may not know to go looking for a flag to enable fallback behaviour for a buggy firmware).

[jonkloske]

As soon as I posted the above I had a sudden memory of a "path" setting in WS View.... maybe there really is a config option for this.

[jonkloske]

So yeah, it may not necessarily be the firmware version (though it’s still generating bad urls), but it may be the change is actually in the WSview app. I finally figured out what you have to set the path setting to if you want it to work:

/path?

(or similar, see attached screenshot)

So yeah, maybe try that in your apps before modifying the code :)

[Alvipas]

Hi Jonkloske, I'll try to verify that.

It may take a few days before I can go to the station site.

Great work!

[jonkloske]

Just remember if you do make the change and you currently have a workaround in your code, it may stop working again (since it’ll be expecting the broken behaviour which would no longer be happening). I had to change both sides at the same time (or deal with it being broken for a little bit).

[Alvipas]

Just remember if you do make the change and you currently have a workaround in your code, it may stop working again (since it’ll be expecting the broken behaviour which would no longer be happening). I had to change both sides at the same time (or deal with it being broken for a little bit).

Hi, I can confirm that your solution is working for me. Thanks again!

[jonkloske]

So looks as though at least in many cases this is a combination of a firmware that lets you generate "invalid" URLs, and a config client change that let you easily specify a path that runs into the firmware issue.

[LapplandsCohan]

Adding the /path? in the device configuration also solved my problem with ecowitt-client mode.

[johlym]

Ambient user checking in. Adding /path? helped with my WS2092.