huge spikes in indoor data

[Eddie-BS]

Hi, I found an other issue ... Perhaps due to the fact that my PWS is rebooting every morning It seems to send invalid data for the INDOOR sensors ....

In the database I see values that are way off. something like -5500 deg for temp and a value like that for pressure ... It is only for the INTERNAL sensors and that might be because of they have to sync with the PWS after an connection reset between the PWS and the internal sensor. Because of this, all combined sensers using this data also show these spikes Is it possible to add some generic check for data "way off" ?

[vaneeten]

Yes, that shouldn't be too hard to add. I will add it.

[Eddie-BS]

I was looking at the moment of the "wrong" data"

These 2 are not read well, probably a "bad" value from the PWS ... This causes

[Eddie-BS]

Hmm, I noticed the huge spikes in almost all sensors last night ... there should be some data validation on all ...

[vaneeten]

Eddie, I think I have the outlier filtering covered. The new version 0.2.0 has been checked in. Because my weatherstation doesn't produce such outliers, I have no quick way to test it but the plugin works like before on my installation. By the way, you can clean-up existing outliers in the database by shift-clicking on the datapoint that you want to remove.

[Eddie-BS]

Hi, I installed 0.2.0 and will let you know if all is OK I allready removed the spikes manually

[Eddie-BS]

Ok, in the webinterface it all looks OK ... In the logging I notice an error right after the sensors are way off ...

2021-06-06 09:47:33.171  WuDirect: (WuDirect) URL CALLED: /weatherstation/updateweatherstation.php?ID=xxxxxx&PASSWORD=xxxxxx&tempf=64.2&humidity=67&dewptf=53.1&windchillf=64.2&winddir=213&windspeedmph=0.00&windgustmph=0.00&rainin=0.00&dailyrainin=0.00&weeklyrainin=0.00&monthlyrainin=0.01&yearlyrainin=13.81&solarradiation=381.91&UV=4&indoortempf=-9999&indoorhumidity=-9999&baromin=-9999&lowbatt=0&dateutc=now&softwaretype=WH2602%20V4.5.7&action=updateraw&realtime=1&rtfreq=5

right after this, it gives an error in the logging ....

2021-06-06 09:47:33.182  Error: WuDirect: (WuDirect) 'onMessage' failed 'TypeError'.
2021-06-06 09:47:33.182  Error: WuDirect: (WuDirect) ----> Line 430 in '/opt/domoticz/userdata/plugins/domoticz-wudirect/plugin.py', function onMessage
2021-06-06 09:47:33.182  Error: WuDirect: (WuDirect) ----> Line 367 in '/opt/domoticz/userdata/plugins/domoticz-wudirect/plugin.py', function onMessage
2021-06-06 09:47:33.182  Error: WuDirect: (WuDirect) ----> Line 487 in '/opt/domoticz/userdata/plugins/domoticz-wudirect/plugin.py', function is_number
2021-06-06 09:47:33.182  WuDirect: (WuDirect) Local context:
2021-06-06 09:47:33.182  WuDirect: (WuDirect) ----> 'CompositeSensors'     '{'THB': {'nr': 100, 'type': 'Temp+Hum+Baro', 'src': ['tempf', 'humidity', '_humstat,humidity,tempf', 'baromin', '_forecast,baromin']}, 'WTC': {'nr': 101, 'type': 'Wind+Temp+Chill', 'src': ['winddir', '_winddir,winddir', '_windms10,windspeedmph', '_windms10,windgustmph', 'tempf', 'windchillf']}, 'Barometer': {'nr': 102, 'type': 'Barometer', 'src': ['baromin', '_forecast,baromin']}, 'Rain': {'nr': 103, 'type': 'Rain', 'src': ['_rain100,rainin', 'dailyrainin']}, 'UV': {'nr': 104, 'type': 'UV', 'src': ['UV', 'tempf']}}'
2021-06-06 09:47:33.182  WuDirect: (WuDirect) ----> 'Devices'              '{1: <Domoticz.Device object at 0x7f96d453b290>, 3: <Domoticz.Device object at 0x7f96d453b588>, 5: <Domoticz.Device object at 0x7f96d453b620>, 6: <Domoticz.Device object at 0x7f96d453b6b8>, 7: <Domoticz.Device object at 0x7f96d453b750>, 8: <Domoticz.Device object at 0x7f96d453b7e8>, 9: <Domoticz.Device object at 0x7f96d453b880>, 10: <Domoticz.Device object at 0x7f96d453b918>, 11: <Domoticz.Device object at 0x7f96d453b9b0>, 14: <Domoticz.Device object at 0x7f96d453ba48>, 15: <Domoticz.Device object at 0x7f96d453bae0>, 16: <Domoticz.Device object at 0x7f96d453bb78>, 18: <Domoticz.Device object at 0x7f96d453bc10>, 19: <Domoticz.Device object at 0x7f96d453bca8>, 100: <Domoticz.Device object at 0x7f96d453bd40>, 101: <Domoticz.Device object at 0x7f96d453bdd8>, 102: <Domoticz.Device object at 0x7f96d453be70>, 103: <Domoticz.Device object at 0x7f96d453bf08>, 104: <Domoticz.Device object at 0x7f96d455f030>}'
2021-06-06 09:47:33.182  WuDirect: (WuDirect) ----> 'Domoticz'             '<module 'Domoticz' (built-in)>'
2021-06-06 09:47:33.182  WuDirect: (WuDirect) ----> 'Images'               '{}'
2021-06-06 09:47:33.182  WuDirect: (WuDirect) ----> 'Parameters'           '{'HardwareID': 3, 'HomeFolder': '/opt/domoticz/userdata/plugins/domoticz-wudirect/', 'StartupFolder': '/opt/domoticz/', 'UserDataFolder': '/opt/domoticz/userdata/', 'WebRoot': '', 'Database': '/opt/domoticz/userdata/domoticz.db', 'Language': 'en', 'Version': '0.2.0', 'Author': 'mveeten', 'Name': 'WuDirect', 'Address': '', 'Port': '8008', 'SerialPort': '', 'Username': '', 'Password': '', 'Key': 'WuDirect', 'Mode1': '', 'Mode2': '', 'Mode3': '', 'Mode4': '', 'Mode5': '', 'Mode6': 'File', 'DomoticzVersion': '2021.1 (build 13277)', 'DomoticzHash': '16492be90', 'DomoticzBuildTime': '2021-05-25 07:52:07'}'
2021-06-06 09:47:33.182  WuDirect: (WuDirect) ----> 'SensorTable'          '{'humidity': {'nr': 1, 'name': 'Humidity Outdoor', 'type': 'Custom', 'scale': 1.0, 'offset': 0.0, 'min': 0, 'max': 100, 'unit': '%'}, 'monthlyrainin': {'nr': 2, 'name': '', 'type': '', 'scale': 1.0, 'offset': 0.0, 'min': 0, 'max': 100000, 'unit': ''}, 'solarradiation': {'nr': 3, 'name': 'Solar', 'type': 'Solar Radiation', 'scale': 1.0, 'offset': 0.0, 'min': 0, 'max': 1380, 'unit': 'W/m^2'}, 'rtfreq': {'nr': 4, 'name': '', 'type': '', 'scale': 1.0, 'offset': 0.0, 'min': -100, 'max': 100, 'unit': ''}, 'dewptf': {'nr': 5, 'name': 'Dewpoint', 'type': 'Temperature', 'scale': 0.5555555555555556, 'offset': -17.77777777777778, 'min': -100, 'max': 100, 'unit': 'C'}, 'baromin': {'nr': 6, 'name': 'Pressure', 'type': 'Pressure', 'scale': 33.86, 'offset': 0.0, 'min': 800, 'max': 1040, 'unit': 'hPa'}, 'tempf': {'nr': 7, 'name': 'Temperature Outdoor', 'type': 'Temperature', 'scale': 0.5555555555555556, 'offset': -17.77777777777778, 'min': -100, 'max': 100, 'unit': 'C'}, 'windspeedmph': {'nr': 8, 'name': 'Wind Speed', 'type': 'Custom', 'scale': 1.609344, 'offset': 0.0, 'min': 0, 'max': 400, 'unit': 'km/h'}, 'windchillf': {'nr': 9, 'name': 'Wind Chill', 'type': 'Temperature', 'scale': 0.5555555555555556, 'offset': -17.77777777777778, 'min': -100, 'max': 100, 'unit': 'C'}, 'indoorhumidity': {'nr': 10, 'name': 'Humidity Indoor', 'type': 'Custom', 'scale': 1.0, 'offset': 0.0, 'min': 0, 'max': 100, 'unit': '%'}, 'rainin': {'nr': 11, 'name': 'Rain Rate', 'type': 'Custom', 'scale': 25.4, 'offset': 0.0, 'min': 0, 'max': 1000, 'unit': 'mm/h'}, 'ID': {'nr': 12, 'name': '', 'type': '', 'scale': 1.0, 'offset': 0.0, 'min': '', 'max': '', 'unit': ''}, 'dateutc': {'nr': 13, 'name': '', 'type': '', 'scale': 1.0, 'offset': 0.0, 'min': '', 'max': '', 'unit': ''}, 'UV': {'nr': 14, 'name': '$UV', 'type': 'Custom', 'scale': 1.0, 'offset': 0.0, 'min': 0, 'max': 50, 'unit': ''}, 'indoortempf': {'nr': 15, 'name': 'Temperature Indoor', 'type': 'Temperature', 'scale': 0.5555555555555556, 'offset': -17.77777777777778, 'min': -100, 'max': 100, 'unit': 'C'}, 'winddir': {'nr': 16, 'name': 'Wind Direction', 'type': 'Custom', 'scale': 1.0, 'offset': 0.0, 'min': -360, 'max': 360, 'unit': 'Deg'}, 'absbaromin': {'nr': 17, 'name': '', 'type': 'Pressure', 'scale': 33.86, 'offset': 0.0, 'min': 800, 'max': 1040, 'unit': ''}, 'windgustmph': {'nr': 18, 'name': 'Wind Gust', 'type': 'Custom', 'scale': 1.609344, 'offset': 0.0, 'min': 0, 'max': 400, 'unit': 'km/h'}, 'dailyrainin': {'nr': 19, 'name': 'Rain Daily', 'type': 'Custom', 'scale': 25.4, 'offset': 0.0, 'min': 0, 'max': 10000, 'unit': 'mm'}}'
2021-06-06 09:47:33.183  WuDirect: (WuDirect) ----> 'Settings'             '{'DB_Version': '148', 'Title': 'Domoticz', 'LightHistoryDays': '30', 'MeterDividerEnergy': '1000', 'MeterDividerGas': '100', 'MeterDividerWater': '100', 'RandomTimerFrame': '15', 'ElectricVoltage': '230', 'CM113DisplayType': '0', '5MinuteHistoryDays': '1', 'SensorTimeout': '60', 'SensorTimeoutNotification': '0', 'UseAutoUpdate': '1', 'UseAutoBackup': '1', 'CostEnergy': '2149', 'CostEnergyT2': '2149', 'CostEnergyR1': '800', 'CostEnergyR2': '800', 'CostGas': '6218', 'CostWater': '16473', 'UseEmailInNotifications': '1', 'SendErrorNotifications': '0', 'EmailPort': '25', 'EmailAsAttachment': '0', 'DoorbellCommand': '0', 'SmartMeterType': '0', 'EnableTabLights': '1', 'EnableTabTemp': '1', 'EnableTabWeather': '1', 'EnableTabUtility': '1', 'EnableTabCustom': '1', 'EnableTabScenes': '0', 'EnableTabFloorplans': '0', 'NotificationSensorInterval': '43200', 'NotificationSwitchInterval': '0', 'RemoteSharedPort': '6144', 'Language': 'en', 'DashboardType': '0', 'MobileType': '0', 'WindUnit': '0', 'TempUnit': '0', 'WeightUnit': '0', 'SecStatus': '0', 'SecOnDelay': '30', 'AuthenticationMethod': '0', 'ReleaseChannel': '1', 'RaspCamParams': '-w 800 -h 600 -t 1', 'UVCParams': '-S80 -B128 -C128 -G80 -x800 -y600 -q100', 'AcceptNewHardware': '1', 'ZWavePollInterval': '60', 'ZWaveEnableDebug': '0', 'ZWaveNetworkKey': '0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0x10', 'ZWaveEnableNightlyNetworkHeal': '0', 'BatteryLowNotification': '0', 'AllowWidgetOrdering': '1', 'ActiveTimerPlan': '0', 'HideDisabledHardwareSensors': '1', 'EnableEventScriptSystem': '1', 'EventSystemLogFullURL': '1', 'DisableDzVentsSystem': '0', 'DzVentsLogLevel': '3', 'LogEventScriptTrigger': '1', 'WebTheme': 'default', 'FloorplanPopupDelay': '750', 'FloorplanFullscreenMode': '0', 'FloorplanAnimateZoom': '1', 'FloorplanShowSensorValues': '1', 'FloorplanShowSwitchValues': '0', 'FloorplanShowSceneNames': '1', 'FloorplanRoomColour': 'Blue', 'FloorplanActiveOpacity': '25', 'FloorplanInactiveOpacity': '5', 'TempHome': '20', 'TempAway': '15', 'TempComfort': '22.0', 'DegreeDaysBaseTemperature': '18.0', 'HTTPURL': 'aHR0cHM6Ly93d3cuc29tZWdhdGV3YXkuY29tL3B1c2h1cmwucGhwP3VzZXJuYW1lPSNGSUVMRDEmcGFzc3dvcmQ9I0ZJRUxEMiZhcGlrZXk9I0ZJRUxEMyZmcm9tPSNGSUVMRDQmdG89I1RPJm1lc3NhZ2U9I01FU1NBR0U=', 'HTTPPostContentType': 'YXBwbGljYXRpb24vanNvbg==', 'ShowUpdateEffect': '0', 'ShortLogInterval': '5', 'SendErrorsAsNotification': '0', 'IFTTTEnabled': '0', 'EmailEnabled': '1', 'MaxElectricPower': '6000', 'Location': '52.26594;5.17162', 'ClickatellEnabled': '0', 'ClickatellAPI': '0', 'ClickatellFrom': '0', 'ClickatellTo': '0', 'EmailFrom': '0', 'EmailServer': '0', 'EmailTo': '0', 'EmailPassword': '0', 'EmailUsername': '0', 'FCMEnabled': '0', 'HTTPEnabled': '0', 'HTTPField1': '0', 'HTTPField2': '0', 'HTTPField3': '0', 'HTTPField4': '0', 'HTTPPostData': '0', 'HTTPPostHeaders': '0', 'HTTPTo': '0', 'KodiIPAddress': '224.0.0.1', 'KodiEnabled': '0', 'KodiPort': '9777', 'KodiTimeToLive': '5', 'LmsPlayerMac': '0', 'LmsDuration': '5', 'LmsEnabled': '0', 'ProwlAPI': '0', 'ProwlEnabled': '0', 'PushALotAPI': '0', 'PushALotEnabled': '0', 'PushbulletAPI': '0', 'PushbulletEnabled': '0', 'PushoverAPI': '0', 'PushoverUser': '0', 'PushoverEnabled': '0', 'PushsaferAPI': '0', 'PushsaferImage': '0', 'PushsaferEnabled': '0', 'TelegramAPI': '0', 'TelegramChat': '0', 'TelegramEnabled': '0', 'WebLocalNetworks': '0', 'WebRemoteProxyIPs': '0', 'SecPassword': 'd41d8cd98f00b204e9800998ecf8427e', 'ProtectionPassword': 'd41d8cd98f00b204e9800998ecf8427e', 'MyDomoticzUserId': '0', 'MyDomoticzSubsystems': '0', 'OneWireSensorPollPeriod': '0', 'OneWireSwitchPollPeriod': '0', 'IFTTTAPI': '0'}'
2021-06-06 09:47:33.183  WuDirect: (WuDirect) ----> '_plugin'              '<plugin.BasePlugin object at 0x7f96d45d9470>'
2021-06-06 09:47:33.183  WuDirect: (WuDirect) ----> 'urlparse'             '<module 'urllib.parse' from '/usr/lib/python3.7/urllib/parse.py'>'

[vaneeten]

Thank you for your detailed report! It should be fixed in 0.2.1.

[Eddie-BS]

running for a while now without issues ! thanks

[vaneeten]

Your feedback is much appreciated!