matthewwall / weewx-interceptor

weewx driver that intercepts web traffic from internet 'bridge' devices such as Acurite Access, ObserverIP, OS LW30x, LaCross GW1000U, FineOffset GW1000
GNU General Public License v3.0
105 stars 46 forks source link

Interceptor in listen mode for AmbientWeather WS-2902C only currently works with THIS tweak #109

Open KruseLuds opened 2 years ago

KruseLuds commented 2 years ago

This attached file is a workaround/fix for an issue with intercepting AmbientWeather console messages using the Interceptor driver for the WS-2902C model weather (and possibly similar models - in listen mode):

The latest interceptor integration has a file which I have renamed interceptor.py.new.bad, a copy is in this .zip file as well for comparison purposes. This ends up causing all of the data sent to various weather web sites to actually be an empty message! The older interceptor.py file which is included here (renamed to interceptor.py.old.good), still works:

old_interceptor.py_that_works.zip

Just rename the interceptor.py.old.good file as interceptor.py and place it in:

"/usr/share/weewx/user" (that last directory is NOT misspelled)

(You might want to rename the old (newer) interceptor.py that is already there on your machine first for your comfort level).

Note, there will be several messages in the log about "unrecognized parameters" when using this driver, but those are the more esoteric weather station details that are typically not used - and those messages can be ignored, unless you want to update weewx and the DB schema to also handle those.

defunctknightsurfer commented 1 year ago

I am going to confirm @KruseLuds findings.

I was running 0.60 and once I replaced it with the above attached 0.54b, it began to work.

Here you can see the log when my 2902 sent data when running .60

Nov 30 12:52:21 jakku weewx[2990271] DEBUG user.interceptor: GET:
Nov 30 12:52:21 jakku weewx[2990271] DEBUG user.interceptor: raw data:
Nov 30 12:52:21 jakku weewx[2990271] DEBUG user.interceptor: raw packet: {'dateTime': 1669841542, 'usUnits': 16}
Nov 30 12:52:21 jakku weewx[2990271] DEBUG user.interceptor: mapped packet: {'dateTime': 1669841542, 'usUnits': 16}

And here you can see after downgrading to 0.54b

Dec  2 12:35:59 jakku weewx[3992791] DEBUG user.interceptor: GET: stationtype=AMBWeatherV4.3.4&PASSKEY=XXXX&dateutc=2022-11-30+23:21:23&tempinf=67.6&humidityin=40&baromrelin=29.735&baromabsin=29.495&tempf=35.2&battout=1&humidity=99&winddir=20&windspeedmph=0.0&windgustmph=0.0&maxdailygust=3.4&hourlyrainin=0.000&eventrainin=0.378&dailyrainin=0.378&weeklyrainin=1.157&monthlyrainin=6.299&totalrainin=42.319&solarradiation=16.81&uv=0&temp1f=65.3&humidity1=49&temp2f=41.9&humidity2=76&batt1=1&batt2=1&batt_co2=1
Dec  2 12:35:59 jakku weewx[3992791] DEBUG user.interceptor: raw data: stationtype=AMBWeatherV4.3.4&PASSKEY=E8:DB:84:E7:22:F8&dateutc=2022-11-30+23:21:23&tempinf=67.6&humidityin=40&baromrelin=29.735&baromabsin=29.495&tempf=35.2&battout=1&humidity=99&winddir=20&windspeedmph=0.0&windgustmph=0.0&maxdailygust=3.4&hourlyrainin=0.000&eventrainin=0.378&dailyrainin=0.378&weeklyrainin=1.157&monthlyrainin=6.299&totalrainin=42.319&solarradiation=16.81&uv=0&temp1f=65.3&humidity1=49&temp2f=41.9&humidity2=76&batt1=1&batt2=1&batt_co2=1
Dec  2 12:35:59 jakku weewx[3992791] DEBUG user.interceptor: using rain_total 0.378 from dailyrainin
Dec  2 12:35:59 jakku weewx[3992791] INFO user.interceptor: unrecognized parameter stationtype=AMBWeatherV4.3.4
Dec  2 12:35:59 jakku weewx[3992791] DEBUG user.interceptor: ignored parameter PASSKEY=XXXXXXXXXXXXXXXXX
Dec  2 12:35:59 jakku weewx[3992791] INFO user.interceptor: unrecognized parameter baromrelin=29.735
Dec  2 12:35:59 jakku weewx[3992791] INFO user.interceptor: unrecognized parameter battout=1
Dec  2 12:35:59 jakku weewx[3992791] INFO user.interceptor: unrecognized parameter maxdailygust=3.4
Dec  2 12:35:59 jakku weewx[3992791] INFO user.interceptor: unrecognized parameter hourlyrainin=0.000
Dec  2 12:35:59 jakku weewx[3992791] INFO user.interceptor: unrecognized parameter eventrainin=0.378
Dec  2 12:35:59 jakku weewx[3992791] DEBUG user.interceptor: ignored parameter weeklyrainin=1.157
Dec  2 12:35:59 jakku weewx[3992791] DEBUG user.interceptor: ignored parameter monthlyrainin=6.299
Dec  2 12:35:59 jakku weewx[3992791] INFO user.interceptor: unrecognized parameter totalrainin=42.319
Dec  2 12:35:59 jakku weewx[3992791] INFO user.interceptor: unrecognized parameter temp1f=65.3
Dec  2 12:35:59 jakku weewx[3992791] INFO user.interceptor: unrecognized parameter humidity1=49
Dec  2 12:35:59 jakku weewx[3992791] INFO user.interceptor: unrecognized parameter temp2f=41.9
Dec  2 12:35:59 jakku weewx[3992791] INFO user.interceptor: unrecognized parameter humidity2=76
Dec  2 12:35:59 jakku weewx[3992791] INFO user.interceptor: unrecognized parameter batt1=1
Dec  2 12:35:59 jakku weewx[3992791] INFO user.interceptor: unrecognized parameter batt2=1
Dec  2 12:35:59 jakku weewx[3992791] INFO user.interceptor: unrecognized parameter batt_co2=1
Dec  2 12:35:59 jakku weewx[3992791] INFO user.interceptor: skipping rain measurement of 0.378: no last rain
Dec  2 12:35:59 jakku weewx[3992791] DEBUG user.interceptor: raw packet: {'dateTime': 1669850483, 'usUnits': 1, 'rain_total': 0.378, 'temperature_in': 67.6, 'humidity_in': 40.0, 'pressure': 29.495, 'temperature_out': 35.2, 'humidity_out': 99.0, 'wind_dir': 20.0, 'wind_speed': 0.0, 'wind_gust': 0.0, 'solar_radiation': 16.81, 'uv': 0.0, 'rain': None}
Dec  2 12:35:59 jakku weewx[3992791] DEBUG user.interceptor: mapped packet: {'dateTime': 1669850483, 'usUnits': 1, 'pressure': 29.495, 'outHumidity': 99.0, 'inHumidity': 40.0, 'inTemp': 67.6, 'windSpeed': 0.0, 'windGust': 0.0, 'windDir': 20.0, 'radiation': 16.81, 'rain': None, 'UV': 0.0}
defunctknightsurfer commented 1 year ago

Going back to the 0.60, I made the following changes and was able to get 0.60 working with my 2902. The changes and code were based off what @KruseLuds found originally.

--- interceptor.py.0.60 2022-12-02 12:29:12.028535021 -0800
+++ interceptor.py.patch    2022-12-02 13:54:57.643921284 -0800
@@ -620,7 +620,14 @@

         def do_GET(self):
             # get the query string from an HTTP GET
-            data = urlparse.urlparse(self.path).query
+            data = self.path if self.path else ''
+            if '?' in data:
+                # if this is a query string, get just the query part
+                data = urlparse.urlparse(data).query
+            idx = data.rfind('/')
+            if idx >= 0:
+                # if this has path in it, strip the path
+                data = data[idx + 1:]
             logdbg('GET: %s' % _obfuscate_passwords(data))
             Consumer.queue.put(data)
             self.reply()
KruseLuds commented 1 year ago

Hello @pkringle , sorry about the super long delay, I have had a chance to revisit this now. Mainly also, because I have found another issue. I have been measuring the length of the interval of the traffic coming from the WS-2902C console - into weewx - in a lazy way - by comparing the "unrecognized" messages appearing in the log. Sometimes they appear every 16 seconds (which is the correct interval) but sometimes they only come across every 32 seconds. Please check to see if this happens to you as well. I believe the missed or skipped messages not appearing in the log are because of another code issue within the interceptor...

I also did examine the old and the new files and I have implemented your suggested changes and my newer version of the interceptor.py does now work as well. As a programmer from wall street for many years, I am almost embarrassed that I am too lazy to really study the code an understand it, so making changes without knowing what I am doing makes me feel like a bull in a china shop. My 15 vs. 32 second issue may force me to really understand what is going on here once I get the chance (I am always so crammed for time!). Also, everywhere I report weather to web sites via weewx, they never want it that often anyway... but us purists hate technology that does not work perfectly.... :-)

In any event, I noticed that you had discovered that same code that was in the old file was missing from the new one... I also noticed just above that code in the new file - is this:

  def do_PUT(self):
        pass

when in fact the older file had instead:

    def do_PUT(self):
        data = urlparse.urlparse(self.path).query
        logdbg('PUT: %s' % _obfuscate_passwords(data))
        self.reply()

Although the system is working with your suggested change (and the newer version may have the "do-PUT(...)" function removed because it is no longer called), I have also made the above update to see if that would have any effect on the 16 vs 32 second bug. It is possible Matt did not actually remove that other function but in fact forgot to include it with his newer build, as that appears to be the case with the other piece of missing code that you had discovered.

Anyway, I've set up a command-line sensor in home assistant measuring the interval in the syslog and here is recent data on the 16 vs. 32 intervals I have noticed (and sometimes it may be slightly different maybe because of other latencies, but I digress). Double-click on it to enlarge it (unfortuinately that makes it a little blurry - but still readable) -

image

Please let me know your thoughts. I look forward to hearing back!

KruseLuds commented 9 months ago

I have updated my file as well - not sure if this is fixed in the latest version, though I am now running 5.0.2 -