ERROR: Error on calculation (lib.calculator)

[Delphius7]

Hi Peter, First of all thanks for your incredible work in describing how gather data from a watermeter. I successfully receive the data from wmbusmeters and now i try to get ha-watermenter working.

Unfortunately I get an exception when running it. I have traced it back to strptime in line 58 of calculator.py and printed the exception, see below: time data '2020-11-14 00:53:58.746042' does not match format '%Y-%m-%dT%H:%M:%SZ'

Full log:

2020-11-14 00:54:20,185 - DEBUG: ✔︎ Message Id 5 published. (__main__:62)
2020-11-14 00:54:29,333 - INFO: ✔︎ MQTT Message received topic tele/wasser/verbrauch (__main__:65)
2020-11-14 00:54:29,334 - INFO: ✔︎ MQTT Publish state topic tele/IZAR/wasser/status (__main__:65)
2020-11-14 00:54:29,334 - DEBUG: Init calculator (lib.calculator:62)
2020-11-14 00:54:29,334 - DEBUG: ✔︎ Load previous data (lib.calculator:62)
2020-11-14 00:54:29,334 - INFO: ✔︎ get previous data from /home/pi/ha-watermeter/ha-watermeter/data/28580778.json (lib.calculator:65)
2020-11-14 00:54:29,334 - DEBUG: ✔︎ init previous data, datafile not found! (lib.calculator:62)
2020-11-14 00:54:29,334 - INFO: ✔︎ Start calculation (lib.calculator:65)
time data '2020-11-14 00:53:58.746042' does not match format '%Y-%m-%dT%H:%M:%SZ'
2020-11-14 00:54:29,334 - ERROR: Error on calculation (lib.calculator:71)
2020-11-14 00:54:29,334 - DEBUG: ✔︎ Message Id 6 published. (__main__:62)

Can you please help me fixing this?

Additionally i noticed that wmbusmeters sends json output in UTC time and not in local time. You seem to use the local timezone of the raspberry. In our case of Austria this means it is 1h off. Isn't this an issue? Or do you convert this somewhere and I have not found it yet?

Thanks!

[zibous]

hi, try the new version with setting (config.py): now you can set your own date and time settings...

# date and time formats
DATEFORMAT_TIMESTAMP = '%Y-%m-%dT%H:%M:%SZ'
DATEFORMAT_CURRENT   = '%Y-%m-%d %H:%M:%S.%f'
DATEFORMAT_HOUR = '%H'
DATEFORMAT_DAY = '%Y-%m-%d'
DATEFORMAT_MONTH = '%Y-%m'
DATEFORMAT_YEAR = '%Y-%m'
TIME_FORMAT = '%H:%M:%S'

Additionally i noticed that wmbusmeters sends json output in UTC time and not in local time. You seem to use the local timezone of the raspberry. In our case of Austria this means it is 1h off. Isn't this an issue? Or do you convert this somewhere and I have not found it yet?

see: https://github.com/weetmuts/wmbusmeters/issues/27

[Delphius7]

Thanks for your help. I just tried it again but unfortunately I get the same error:

2020-11-14 11:48:19,888 - INFO: ✔︎ Start calculation (lib.calculator:65)
2020-11-14 11:48:19,888 - ERROR: Error on calculation (lib.calculator:71)
2020-11-14 11:48:19,888 - DEBUG: ✔︎ Message Id 8 published. (__main__:62)

It seems like my issue is with the conversion with strptime in line 58: previous_dts = datetime.strptime(self.pd['timestamp'], DATEFORMAT_TIMESTAMP)

Error message: time data '2020-11-14 11:47:20' does not match format '%Y-%m-%dT%H:%M:%SZ'

here the contents of the two variables: (for the first one strptime works, for the second one no)

self.payload['timestamp']: 2020-11-14T10:48:19Z
self.pd['timestamp']:      2020-11-14 11:47:20

btw there is also the 1h time difference again. How do you deal with this when you calculate the hourly m3?

I have also tried a workaround and set: DATEFORMAT_CURRENT = '%Y-%m-%dT%H:%M:%SZ' Then it works just perfectly.

Do you have an idea what is different on my machine? How can I make it working with the standard setting?

[zibous]

hi,

It seems like my issue is with the conversion with strptime in line 58: previous_dts = datetime.strptime(self.pd['timestamp'], DATEFORMAT_TIMESTAMP)

Error message: time data '2020-11-14 11:47:20' does not match format '%Y-%m-%dT%H:%M:%SZ'

previous_dts is from the DATAFILE os.path.join(os.path.dirname(__file__),'data/' + SMARTMETER_ID + ".json"). This file holds the previous data and if the timestamp do not match to the DATEFORMAT_TIMESTAMP you will get this error. If you change the DATEFORMAT settings, delete this file and try again.

How do you deal with this when you calculate the hourly m3?

see: # calculate the m3 (calculator.py)

                ## m3  = currentM3  - previousM3
                self.cd['m3']['current'] = round(meterM3 - float(self.pd['total_m3']), 3)
                self.cd['m3']['hour'] = round(meterM3 - float(self.pd['last_total']['hour_m3']), 3)
                self.cd['m3']['day'] = round(meterM3 - float(self.pd['last_total']['day_m3']), 3)
                self.cd['m3']['month'] = round(meterM3 - float(self.pd['last_total']['month_m3']), 3)
                self.cd['m3']['year'] = round(meterM3 - float(self.pd['last_total']['year_m3']), 3)

Simply read the previous data and calc the values with the current data

btw there is also the 1h time difference again. This is o.k. because i use UTC time not local time, because wmbusmeters sends json output in UTC time and not in local time. UTC time causes fewer problems, as there is no collision when changing the time (summer or winter time).

But the payload holds both, UTC is timestamp and the local time is last_update.

{
    "name": "Wasserz\u00e4hler Haus",
    "device": "watermeter",
    "deviceid": "43430778",
    "date": "2020-11-14",
    "time": "12:06:02",
    "total_m3": 201.548,
    "m3": {
        "current": 0.0,
        "hour": 0.0,
        "day": 0.122,
        "month": 3.357,
        "year": 83.818
    },
    "liter": {
        "current": 0.0,
        "hour": 0.0,
        "day": 122.0,
        "month": 3357.0,
        "year": 83818.0
    },
    "last_total": {
        "hour": "12",
        "hour_m3": 201.548,
        "day": "2020-11-14",
        "day_m3": 201.426,
        "month": "2020-11",
        "month_m3": 198.191,
        "year": "2020-01-01",
        "year_m3": 117.73
    },
    "alarm": "no_alarm",
    "last_alarm": "no_alarm",
    "periode": "2020-11-14",
    "month": "2020-11",
    "year": "2020",
    "timestamp": "2020-11-14T12:06:02Z",
    "last_update": "2020-11-14 13:06:02.930670",
    "elapsed_time": "0:00:08",
    "icon": "mdi:billboard",
    "unit_of_measurement": "\u33a5",
    "version": "1.0.0",
    "attribution": "Data provided by Peter Siebler",
    "data_provider": "..."
}

[Delphius7]

Thank you very much for the detailed explanation and your time. With your help I was able to get it working.