johanmeijer / grott

Growatt inverter monitor
https://github.com/johanmeijer/grott/wiki
395 stars 109 forks source link

The timestamp for records in pvoutput are ahead by 5 hours #28

Closed benpeart closed 3 years ago

benpeart commented 3 years ago

I noticed that the records sent to pvoutput appear to be using the incorrect timezone. In the log file, I saw that "no or no valid time/date found, grott server time will be used". My docker host machine was set to timezone UTC but the time/date stamps on the records in pvoutput were ahead by 5 hours (I'm in timezone "America/New_York" which is -0500).

johanmeijer commented 3 years ago

At this moment I use the servertime (container time) if no time is available in the Growatt data record (appertenly your inverter/shinelink does not put date/time in the records).

Is it an idea to specify timezone with your grott container (-e tz=US or -e tz=America/New_York)?

benpeart commented 3 years ago

I'm unsure why my shinelink isn't setting the date/time in the records. When I sign into it's web server, it has a date/time though I'm not sure how it got it. I noticed it was off by an hour but we just exited daylight savings time so that may explain why. I also noticed that I can't manually set it from the UI - it appears to succeed but when it loads it still shows the same (off by an hour) time. I ran out of ideas on how to get it to embed the correct time and moved on.

I read through the pvoutput API spec looking for anything related to timezone and it isn't very clear to me but it appears pvoutput is expecting the time to be uploaded in local time (rather than UTC). While it is typically simpler if all time be specified in UTC and only converted to local time by UI displaying it to the user; it looks like they went with local time because some simple devices that are reporting data don't know the UTC time.

I saw some discussion about making the date/time optional and if it was left blank, they would just fill in the current time but I can't find if they actually support that or if it was a potential future addition. If they did support it, that could be the simplest solution as you could just leave it blank and let them fill in the date/time. That would allow you to drop your logic to do the same and you also don't have to deal with timezone issues at all. If that doesn't work, there are other potential solutions to try...

Since docker containers are UTC by default, I thought I could just set my timezone in pvoutput to GMT and perhaps that would correct the time but apparently they don't convert things to UTC but just keep them in local time as I didn't see any changes. Everything stayed 5 hours ahead.

I thought about your suggestion to specify the local timezone in the docker container (they are all UTC by default) but that means an additional step for you to document and for each person who uses it to properly configure when they build their docker image. The simplest way I found is using optional environment variables (see "Setting the Timezone" at https://www.ivankrizsan.se/2015/10/31/time-in-docker-containers/) which works with direct docker commands as well as docker-compose. I tested this out manually using the updated steps as of Oct 30, 2016 that include creating the link using 'ln' and am now getting the correct timestamp on the data flowing to pvoutput.

Let me know if you would like me to create a patch/pull request with the changes using the method specified at https://www.ivankrizsan.se/2015/10/31/time-in-docker-containers/.

johanmeijer commented 3 years ago

Isn't enough (and simple) to specify the container timezone at container startup?

It is a default feature of docker to specify the time zone with adding -e tz="your timezone" at the run command (in the YAML file).

I do not mind to create a timezone parameter but I am not sure it is necessary.

benpeart commented 3 years ago

Yes, that makes complete sense. I didn't realize that was a built in feature of docker. It sure makes it simpler!

Setting the timezone this isn't something typically needed in docker so you should document that pvoutput requires you to set the timezone in the docker container to your local timezone using the either docker command line or the docker-compose.yml file.

Note, docker-compose is very specific, it has to be upper case TZ and no quotes around the string. Here is the format that worked for me:

environment:
   - TZ=America/New_York
harryverkooijen commented 3 years ago

I have also problems with the time in docker. I tried above solution but in my docker application (on synology) I can not access the YAML file, is it possible to put the grott YAML file in a sharable volume for easy editing?

benpeart commented 3 years ago

I have a Synology as well. They have UI that allows you to set environment variables for a docker container. In the Synology UI for editing the container, choose the "Environment" tab and add a variable named TZ (case sensitive) with the correct timezone value. You can find a list of timezones here https://en.wikipedia.org/wiki/List_of_tz_database_time_zones

johanmeijer commented 3 years ago

Can we close this one?

benpeart commented 3 years ago

I think the only work to come out of it is to update the docker wiki page to let everyone know 1) that the timezone has to be set to local time and 2) how to set it. I'd also recommend to update the sample docker-compose.yml file give setting the timezone correctly is required.

johanmeijer commented 3 years ago

I updated the wiki and the source YAML file (will be uploaded with the next update).