Closed sermayoral closed 3 years ago
I've noticed something similar as of 0.118, but I think the issue might go beyond Python.
I have a command_line
sensor that runs a bash script including the instruction printf '%(%Y-%m-%d %H:%M:%S%z)T' -2
, which gives the time the shell was invoked. Until I upgraded, it always gave the local time.
When Home Assistant (supervised) runs the script to update the sensor, it gives the current UTC time. For example, 2020-11-20 02:57:02+0000
. If I log in to the SSH container and run the script myself, though, it gives the current local time. For example, 2020-11-19 20:57:02-0600
. In both cases, the TZ
environment variable is set to my local time zone.
If there were a Python problem with datetime
, I wouldn't expect the shell to also have the problem. Perhaps it's a deeper environmental issue instead.
Same behavior here!
Are you sure your locale is correct?
Even with your example printf '%(%Y-%m-%d %H:%M:%S%z)T' -2
I get the output in my TZ (both in the homeassistant container, and in the SSH container)
The datetime.datetime.now() function in my Python script also returns UTC instead of local time (+6 hours in my case). That function returned local time prior to upgrading to HA 0.118.
@tggman tracked in #43412
@ludeeus in my case, yes, I have my locale set to Europe/Madrid.
Home Assistant Logs Registry show me the right local time. Supervisor logs show me UTC time (it has always been in that way, it's not new), but python_script , after 0.118, give me the UTC time, so all my python_scripts that use datetime.datetime.now() are not working propertly
fwiw, because of that behavior, I've always used (/had to use) this in all of my python scripts, and this goes way back. I see no change in python scripts using datetime.datetime.now()
at all since 118
utc_offset = hass.states.get('sensor.utc_offset').state
timeDifference = float(utc_offset)
if (state.state == filter or debug):
dt = state.last_changed + datetime.timedelta(hours= timeDifference)
time = '%02d:%02d' % (dt.hour,dt.minute)
# If state changed in the past days show the date too
if dt.date() < datetime.datetime.now().date():
time = '{} {}'.format('%02d/%02d' % (dt.day,dt.month),time)
with the sensor.utc_offset
being
utc_offset:
friendly_name: Utc offset
value_template: >
{{now().utcoffset().total_seconds()/3600}}
@sermayoral can you try running ha core rebuild
?
Marius is correct. datetime.datetime.now()
is not tied into the timezone set in Home Assistant. It is tied to your system timezone.
Your system timezone should be set by Home Assistant if you run Supervised / OS. If that is wrong, ha core rebuild
as Ludeeus suggests, should set it.
Also make sure the timezone is right when you go to config -> General.
@ludeeus sure!
➜ /workspace hassio homeassistant rebuild
⣾ Processing... Done.
Command completed successfully.
The process has finished, Home Assistant restarted but same issue :-(
@balloob I'm running Supervised, so my timezone should be set by Home Assistant. ha core rebuild
didn't work :-(
My timezone is right in Config -> General:
A logger output of a python_script [logger.warning('datetime.datetime.now() output: {}'.format(datetime.datetime.now()))
]:
System timezone is right in the Home Assistant Environment:
➜ /workspace date
Fri Nov 20 16:15:14 CET 2020
System timezone is right in the Debian OS as well:
$ date
vie nov 20 16:18:22 CET 2020
@Mariusthvdb I'm pretty sure datetime.datetime.now()
has been returning the local time (right timezone) until I updated to 0.118. At least in my case...
well, ymmv, but in my setup I've been struggling with that ever since (or even sometime before) this
had a hard coded +1 there, which obviously didn't help during DS time periods. So finally decided to use the Utc sensor. Which fixed it ever since. Using OS, (Hassio at the time), if that's of any relevance.
@Mariusthvdb My mileage vary. Sorry.
I'm sorry for your problems more than 3 years ago, but there is anything wrong with python datetime in 0.118 Home Assistant system. Issue that didn't happen in 0.117-. @tggman and @DendelX seem to have the same issue. Something has been broken.
It's 18:24 right now in my country (local time). I have tried the same function in all the PCs in my home, different OS, etc. All of them are returning the right local time:
python test.py
Windows 10 Python 3.7 datetime.datetime.now(): 2020-11-20 18:27:15.408757
With the images attached above, it is shown that the timezone is correctly set, both on the Home Assistant system and on the Debian system. But python datetime is not working as expected.
Additionally. In the Home Assistant System, I have checked this:
The python 2.7 installed in the HA system gives me the correct local time! So, it's clear the HA system has the timezone right. So it seems to affect python_script integration only.
But this is not working well, neither with time.localtime()
, sorry:
Sorry I had a mistake with the keyboard :-(
@balloob I think this is the PR that has caused this issue: https://github.com/home-assistant/core/pull/42146
@sermayoral What do you see if you press "LOAD FULL HOME ASSISTANT LOG" to see the raw logs in that last image?
@amelchio in the full log i can see UTC time:
2020-11-20 17:43:03 WARNING (SyncWorker_47) [homeassistant.components.python_script.gestor_climatizacion.py] time.localtime(): time.struct_time(tm_year=2020, tm_mon=11, tm_mday=20, tm_hour=17, tm_min=43, tm_sec=3, tm_wday=4, tm_yday=325, tm_isdst=0)
This test passes on 117, 118 and 119dev.
async def test_datetime_now(hass, caplog):
"""Test time.sleep warns once."""
caplog.set_level(logging.WARNING)
now = dt_util.now().replace(microsecond=0, tzinfo=None)
source = """
hass.states.set('bla.bla', datetime.datetime.now().replace(microsecond=0).isoformat())
"""
hass.async_add_executor_job(execute, hass, "test.py", source, {})
await hass.async_block_till_done()
assert hass.states.get("bla.bla").state == now.isoformat()
@balloob yeah, the time is the same, but they are shown in different timezones:
@balloob can you do a test that compare dt_util.now().hour and datetime.datetime.now().hour on 117, 118 and 119? But it must be restricted_python datetime classes.
Well, I think I already understand everything. The problem is not the system timezone. I have also tried doing a {{now ()}} in the template engine, and it also returns the local time (with the correct timezone).
The problem only happens with the datetime you use inside python_scripts. This datetime does not have the defined system timezone associated with it (for this reason it shows the UTC time instead of the local time).
The funny thing is why the datetime that I used inside the python_scripts showed me the local time before 0.118.0? Quite a mystery ... Perhaps it is produced by the changes of https://github.com/home-assistant/core/pull/42146, but as much as I look at the PR, I do not see anything strange in it.
I think the solution to this issue is to set the timezone in the time module of restricted_python. Something like this:
In https://github.com/home-assistant/core/blob/dev/homeassistant/components/python_script/__init__.py
set:
>>> os.environ['TZ'] = 'Europe/London' (value of the timezone set in config -> general)
>>> time.tzset()
But i'm not sure how to do this :-(
The problem is not the system timezone.
For Home Assistant "the system" is the Docker container. I believe all your findings are consistent with the TZ
environment variable not being properly set inside the container.
@amelchio Thanks. I undestand, but two points about it:
Can you run docker exec homeassistant printenv | grep TZ
?
docker exec homeassistant printenv | grep TZ
@ludeeus sure!
$ docker exec homeassistant printenv | grep TZ
TZ=Europe/Madrid
Well, that's good! but does not explain anything since that should be used for datetime.datetime.now()
Did you list all your configured integrations somewhere (including custom_component)? if not can you list them here?
@ludeeus I swear I have been using datetime.datetime.now()
for a long time and it has always given me the correct time (my heating system depends on this command, together with time.tm_isdst
). As soon as I upgraded to 0.118.0, this problem started.
We are trying to find out what's wrong, nobody is saying you broke it.
Can you try adding this sensor to verify that TZ
is also correct in the running Home Assistant process:
sensor:
- platform: command_line
name: timezone
command: '/bin/sh -c "printenv|grep TZ="'
It should give the same TZ=Europe/Madrid
value.
And this one would be interresting:
docker exec homeassistant python3 -c "import datetime; print(datetime.datetime.now())"
And this one would be interresting:
docker exec homeassistant python3 -c "import datetime; print(datetime.datetime.now())"
@ludeeus here you are:
docker exec homeassistant python3 -c "import datetime; print(datetime.datetime.now())"
2020-11-21 11:17:10.436357
This time does not have the timezone.
Something seems to overwrite how it does that.
This is from inside my homeassistant
container:
bash-5.0# export TZ=Europe/Riga
bash-5.0# python3 -c "import datetime; print(datetime.datetime.now())"
2020-11-21 13:28:06.134757
bash-5.0# export TZ=Europe/Madrid
bash-5.0# python3 -c "import datetime; print(datetime.datetime.now())"
2020-11-21 12:28:44.161060
UTC would be 11:28 at the point of posting this.
Can you try what @amelchio sugested? And this as well:
docker exec homeassistant python3 -m pip freeze
We are trying to find out what's wrong, nobody is saying you broke it.
Can you try adding this sensor to verify that
TZ
is also correct in the running Home Assistant process:sensor: - platform: command_line name: timezone command: '/bin/sh -c "printenv|grep TZ="'
It should give the same
TZ=Europe/Madrid
value.
Something seems to overwrite how it does that.
This is from inside my
homeassistant
container:bash-5.0# export TZ=Europe/Riga bash-5.0# python3 -c "import datetime; print(datetime.datetime.now())" 2020-11-21 13:28:06.134757 bash-5.0# export TZ=Europe/Madrid bash-5.0# python3 -c "import datetime; print(datetime.datetime.now())" 2020-11-21 12:28:44.161060
UTC would be 11:28 at the point of posting this.
Can you try what @amelchio sugested? And this as well:
docker exec homeassistant python3 -m pip freeze
What OS is running on the host, and is that 32 or 64 bit?
cat /etc/os-release
uname -a
Can you run this to see if now
has been overridden:
docker exec homeassistant python3 -c "import datetime; print(datetime.datetime.now)"
The output should be something like <built-in method now of type object at 0x92ab80>
(the "at" part will differ)
Can you run this to see if
now
has been overridden:docker exec homeassistant python3 -c "import datetime; print(datetime.datetime.now)"
The output should be something like
<built-in method now of type object at 0x92ab80>
(the "at" part will differ)
$ docker exec homeassistant python3 -c "import datetime; print(datetime.datetime.now)" <built-in method now of type object at 0xb6ac00d4>
What OS is running on the host, and is that 32 or 64 bit?
cat /etc/os-release uname -a
32 bits both, OS and HA
$ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
$ uname -a
Linux raspberrypi4 5.4.72-v7l+ #1356 SMP Thu Oct 22 13:57:51 BST 2020 armv7l GNU/Linux
Issue comes from: https://github.com/alpinelinux/docker-alpine/issues/117
Thanks @pvizeli.
This reference makes more sense, and that explains why this is working on some systems and no on others. The key was the HA architecture version. People using 64 bits architecture version doesn't have this issue, and people using 32 bits version does.
@balloob this is probably the base problem in https://github.com/home-assistant/core/issues/43337 as well
i fix the problem by remove environment : TZ , and add volume
@sermayoral i used with 64-bit and have the same problem
Thank you guys 😊
I can confirm it's working again :-)
The problem
datetime.datetime.now () function, before 0.118, it gave me the local time. After 0.118 i get an hour less (UTC). Seems to be a python datetime issue with timezones.
Environment
Problem-relevant
configuration.yaml
Traceback/Error logs
Additional information
It's a problem similar to https://github.com/home-assistant/core/issues/43337. Looking at the history, I have observed that, since I updated Home Assistant to 0.118, I am experiencing the reported problem.
@balloob I have tagged you as you asked me :-). Thanks!