home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
71.89k stars 30.12k forks source link

After upgrade to 2022.11.1 google calendar starttime is an hour earlier #81527

Closed MagieMalone closed 1 year ago

MagieMalone commented 1 year ago

The problem

Google calendar start time seems to have become an hour earlier, while the calendar entry has not changed, after installing 2022.11.1

What version of Home Assistant Core has the issue?

core-2022.11.1

What was the last working version of Home Assistant Core?

core-2022.10.6

What type of installation are you running?

Home Assistant OS

Integration causing the issue

google calendar

Link to integration documentation on our website

https://www.home-assistant.io/integrations/google/

Diagnostics information

No response

Example YAML snippet

template:
  - sensor:
      - name: calc_leave_time_alarm
        unique_id: "e9dce637-9b57-4321-8b95-10e2b9e56411"
        device_class: "timestamp"
        state: >
          {%- if state_attr('calendar.alarm', 'message')  is search('#bike', ignorecase=true) %}
            {%- if states('sensor.here_travel_time_alarm_bike_duration') == None %}
              {%- set offset = timedelta(minutes=0) %}
            {%- else %}
              {%- set offset = timedelta(minutes=states('sensor.here_travel_time_alarm_bike_duration')|int) %}
            {%- endif %}
          {%- else %}
            {%- if states('sensor.here_travel_time_alarm_duration_in_traffic') == None %}
              {%- set offset = timedelta(minutes=5) %}
            {%-  else %}
              {%- set offset = timedelta(minutes=states('sensor.here_travel_time_alarm_duration_in_traffic')|int +5)  %}
            {%- endif %} 
          {%- endif %}
          {{- (state_attr('calendar.alarm', 'start_time')| as_datetime - offset).astimezone()}}

Anything in the logs that might be useful for us?

no.

Additional information

Time jumped an hour after installing 2022.11.1, however, release notes don't mention a breaking change.

I copy the attribute into a template sensor (see yaml snippet) and this show the change to an hour earlier:

image image

Also in developer tools the time is an hour earlier then the actual calendar entry: {{state_attr('calendar.alarm', 'start_time')}}

Result type: string 2022-11-07 05:30:00

home-assistant[bot] commented 1 year ago

Hey there @allenporter, mind taking a look at this issue as it has been labeled with an integration (google) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `google` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Change the title of the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign google` Removes the current integration label and assignees on the issue, add the integration domain after the command.

(message by CodeOwnersMention)


google documentation google source (message by IssueLinks)

tunisiano187 commented 1 year ago

Hello, Same for me, i'm currently making tests, but a strange thing is that with my new events, it's not the case... i'll continue my searchs, but... interested of course if someone else find the solution earlier ;-)

blair287 commented 1 year ago

Same issue here.

FL100 commented 1 year ago

Idem. When I see the calendars in HA all seem OK except the calendar that I use for this integration. In the later all items are shifted an hour earlier after the new HA release was installed.

sebfar9172 commented 1 year ago

Hello I confirm the problem but it's not for all my events

realthk commented 1 year ago

Same here, some events fine, but most are an hour early since the DST period end (I don't think it has anything to do with 2011.11.1, I reckon the bug might have been already there with 2011.10.x - though not sure, we weren't home between the time change and 11.1), and all of these (recurring events) were added weeks, months ago.

In Google events follow each other hourly, except for a lunch break: ha_2

But in HA most of those an hour early: ha_1

blair287 commented 1 year ago

Same here, some events fine, but most are an hour early since the DST period end (I don't think it has anything to do with 2011.11.1, I reckon the bug was already there with 2011.10.x), though of course all of these were added weeks, months ago.

In Google events follow each other hourly, except for a lunch break: ha_2

But in HA most of those an hour early: ha_1

Mine was working fine yesterday even after the clock change only had issue after update to 2022.11

blair287 commented 1 year ago

Screenshot_20221104_084404_Home Assistant Screenshot_20221104_084447_Calendar

Time shifted hour earlier in HA these are reoccurring events. My one off event seem to still be correct.

titiviking commented 1 year ago

On top of the mentioned issue, I'm also facing the issue the the calendar in HA is not getting updated anymore... I've removed events from google that remain visible/present in HA calendar... Anyone having this behaviour as well ?

21Development commented 1 year ago

I am having the same issue of calendar offset, here’s what I have identified so far:

marc115uk commented 1 year ago

Same here seemed to break with the update to 2022.11.0 was correct the day before I upgraded. Seems to report correctly with any new event added but incorrectly with existing events

MagieMalone commented 1 year ago

made a new recurring appointment, but that works correctly indeed, the same as a new single event. It's just the old already planned (in my case) recurring events. I can confirm it's not DST end, as the events on monday - thursday all went fine, only after I Installed 2022.11.1 yesterday the issue began.

allenporter commented 1 year ago

Calendar had a large rewrite to sync and serve from local so that's definitely the issue. Wasn't a breaking change, but a bug here.

Thanks for the reports.

21Development commented 1 year ago

Due to some other issues, I've just restored my 2022.11.0 backup, checked the calendar and it was still the same. Proceeded to restore my 2022.10.5 backup and my calendar entries are now showing the correct time.

(I previously tried disabling the calendar(s) restarting HA 2022.11.1 and then reenabling them which didn't solve the issue)

allenporter commented 1 year ago

Can someone give an example timezone this happens? Also curious if home assistant is in the same time zone or different time zone from the Google calendar timezone.

Having some of the raw debug from gcal_sync.auth for one example event could help though may be private info that would need stripping.

21Development commented 1 year ago

I am in GMT - HA is set to GMT and Google shows the correct times when logging in directly

MagieMalone commented 1 year ago

I'm on GMT+1, set both in Google calendar ( Central European Time - Amsterdam) and in HA (Europe/Amsterdam) and I see the issue.

sebfar9172 commented 1 year ago

i am in GMT+1 HA is set to GMT+1 and google GMT+1

realthk commented 1 year ago

I'm in GMT+1 zone, both in Google an HA. Here is debug log from gcal_sync.auth. Sorry for it is quite long, as I have many calendars in Google, but I did not want to strip out anything that might be important. The only one used in HA is with its etag replaced to "zzzzzzzzzzzzzzz" in this log: 2022-11-04 13:41:44.793 DEBUG (MainThread) [gcal_sync.auth] response={'kind': 'calendar#calendarList', 'etag': '"p33sa17ufoaafm0g"', 'nextSyncToken': 'xxxxxxxxxxxxxx', 'items': [{'kind': 'calendar#calendarListEntry', 'etag': '"xxxxxxxx"', 'id': 'p#weather@group.v.calendar.google.com', 'summary': 'Időjárás', 'description': 'A lakóhelynek megfelelő napi időjárás-előrejelzés', 'timeZone': 'Europe/Budapest', 'summaryOverride': 'Weather', 'colorId': '15', 'backgroundColor': '#9fc6e7', 'foregroundColor': '#000000', 'selected': True, 'accessRole': 'reader', 'defaultReminders': [], 'conferenceProperties': {'allowedConferenceSolutionTypes': ['hangoutsMeet']}}, {'kind': 'calendar#calendarListEntry', 'etag': '"xxxxxxx"', 'id': 'realthk@gmail.com', 'summary': 'realthk@gmail.com', 'timeZone': 'Europe/Budapest', 'colorId': '14', 'backgroundColor': '#9fe1e7', 'foregroundColor': '#000000', 'selected': True, 'accessRole': 'owner', 'defaultReminders': [{'method': 'popup', 'minutes': 10}, {'method': 'email', 'minutes': 10}], 'notificationSettings': {'notifications': [{'type': 'eventCreation', 'method': 'email'}, {'type': 'eventChange', 'method': 'email'}, {'type': 'eventCancellation', 'method': 'email'}, {'type': 'eventResponse', 'method': 'email'}]}, 'primary': True, 'conferenceProperties': {'allowedConferenceSolutionTypes': ['hangoutsMeet']}}, {'kind': 'calendar#calendarListEntry', 'etag': '"xxxxxxxx"', 'id': 'xxxxxxxx4@group.calendar.google.com', 'summary': 'Fejlesztők', 'description': 'DM-os fejlesztői naptár', 'timeZone': 'Europe/Budapest', 'colorId': '15', 'backgroundColor': '#9fc6e7', 'foregroundColor': '#000000', 'selected': True, 'accessRole': 'owner', 'defaultReminders': [], 'conferenceProperties': {'allowedConferenceSolutionTypes': ['hangoutsMeet']}}, {'kind': 'calendar#calendarListEntry', 'etag': '"xxxxxxx"', 'id': 'hu.hungarian#holiday@group.v.calendar.google.com', 'summary': 'Ünnepnapok – Magyarország', 'description': 'Ünnepnapok és hagyományok – Magyarország', 'timeZone': 'Europe/Budapest', 'summaryOverride': 'Ünnepnapok – Magyarország', 'colorId': '7', 'backgroundColor': '#42d692', 'foregroundColor': '#000000', 'selected': True, 'accessRole': 'reader', 'defaultReminders': [], 'conferenceProperties': {'allowedConferenceSolutionTypes': ['hangoutsMeet']}}, {'kind': 'calendar#calendarListEntry', 'etag': '"xxxxxxx"', 'id': 'familyxxxxxxx@group.calendar.google.com', 'summary': 'Családi', 'timeZone': 'UTC', 'colorId': '17', 'backgroundColor': '#9a9cff', 'foregroundColor': '#000000', 'selected': True, 'accessRole': 'writer', 'defaultReminders': [{'method': 'popup', 'minutes': 10}, {'method': 'email', 'minutes': 10}], 'conferenceProperties': {'allowedConferenceSolutionTypes': ['hangoutsMeet']}}, {'kind': 'calendar#calendarListEntry', 'etag': '"xxxxxxx"', 'id': 'xxxxxxxg@group.calendar.google.com', 'summary': 'Telefonhívások', 'timeZone': 'Europe/Budapest', 'colorId': '20', 'backgroundColor': '#cabdbf', 'foregroundColor': '#000000', 'selected': True, 'accessRole': 'owner', 'defaultReminders': [], 'conferenceProperties': {'allowedConferenceSolutionTypes': ['hangoutsMeet']}}, {'kind': 'calendar#calendarListEntry', 'etag': '"1623307964726000"', 'id': 'en.hungarian#holiday@group.v.calendar.google.com', 'summary': 'Holidays in Hungary', 'description': 'Holidays and Observances in Hungary', 'timeZone': 'Europe/Budapest', 'summaryOverride': 'Holidays in Hungary', 'colorId': '8', 'backgroundColor': '#16a765', 'foregroundColor': '#000000', 'selected': True, 'accessRole': 'reader', 'defaultReminders': [], 'conferenceProperties': {'allowedConferenceSolutionTypes': ['hangoutsMeet']}}, {'kind': 'calendar#calendarListEntry', 'etag': '"1646341642788000"', 'id': 'rkhhbddeifkbcik8pst5ipsq4jqj4s36@import.calendar.google.com', 'summary': 'https://www.f1calendar.com/download/f1-calendar_q_gp_alarm-20.ics?t=1582025642623', 'timeZone': 'UTC', 'colorId': '8', 'backgroundColor': '#16a765', 'foregroundColor': '#000000', 'selected': True, 'accessRole': 'reader', 'defaultReminders': [], 'conferenceProperties': {'allowedConferenceSolutionTypes': ['hangoutsMeet']}}, {'kind': 'calendar#calendarListEntry', 'etag': '"xxxxxxxx"', 'id': 'xxxxxxxxxx@group.calendar.google.com', 'summary': 'Családi események', 'timeZone': 'Europe/Budapest', 'colorId': '3', 'backgroundColor': '#f83a22', 'foregroundColor': '#000000', 'selected': True, 'accessRole': 'owner', 'defaultReminders': [], 'conferenceProperties': {'allowedConferenceSolutionTypes': ['hangoutsMeet']}}, {'kind': 'calendar#calendarListEntry', 'etag': '"zzzzzzzzzzzzzzz"', 'id': 'zzzzzzzzzzzzzzz@googlemail.com', 'summary': 'zzzzzzzzzzzzzzz@googlemail.com', 'timeZone': 'Europe/Budapest', 'colorId': '5', 'backgroundColor': '#ff7537', 'foregroundColor': '#000000', 'selected': True, 'accessRole': 'reader', 'defaultReminders': [], 'conferenceProperties': {'allowedConferenceSolutionTypes': ['hangoutsMeet']}}, {'kind': 'calendar#calendarListEntry', 'etag': '"1667564698024000"', 'id': '5s59gaimrq1e2u9ej2qbeoru2o@group.calendar.google.com', 'summary': 'Magyar névnapok', 'timeZone': 'Europe/Budapest', 'colorId': '11', 'backgroundColor': '#fbe983', 'foregroundColor': '#000000', 'selected': True, 'accessRole': 'owner', 'defaultReminders': [], 'conferenceProperties': {'allowedConferenceSolutionTypes': ['hangoutsMeet']}}]}

allenporter commented 1 year ago

@realthk thanks that helps. I realize the local calendar event data is also stored in config/.storage/google.xxxxx so maybe you can also snag one event record that shows the problem and let me know the expected time and actual time.

Thank you all, hopefully we can get a quick fix out once I can reproduce.

allenporter commented 1 year ago

Seems like the most likely thing for me to explore to reproduce is for a recurring event that crosses the daylight savings time boundary.

realthk commented 1 year ago

Exactly, @allenporter. The file from Google in .storage is a mess for human eyes, events with the same name occuring several times, I have to find the latest in effect, but it seems the problem is with crossing the DST boundary with a recurring event indeed.

For example, from my screenshot above "Isti" should be at 13:00, and because it was created in september, still during DST, it got GMT+2, which isn't valid anymore, after DST has ended, it should be GMT+1 now (but do not understand then why it works fine for those who restore HA version 2022.10.x...

            "1ookvdm08gqvkheintf7u8folo": {
              "id": "1ookvdm08gqvkheintf7u8folo",
              "ical_uuid": null,
              "summary": "Isti",
              "start": {
                "date": null,
                "date_time": "2022-09-09T13:00:00+02:00",
                "timezone": "Europe/Budapest"
              },
              "end": {
                "date": null,
                "date_time": "2022-09-09T14:00:00+02:00",
                "timezone": "Europe/Budapest"
              },
              "description": null,
              "location": null,
              "transparency": "opaque",
              "status": "confirmed",
              "event_type": "default",
              "visibility": "default",
              "attendees": [],
              "attendees_omitted": false,
              "recurrence": [
                "RRULE:FREQ=WEEKLY;BYDAY=FR"
              ],
              "recurring_event_id": null,
              "original_start_time": null
            },
allenporter commented 1 year ago

Thank you for confirming, i'll get started on trying to reproduce this with a test and working on a fix. No other data needed from users now, thank you all for your patience.

allenporter commented 1 year ago

Also to answer your question about 2022.10 vs 2022.11 it's because I added a bug. The context is because in preparation for being able to create/delete calendar events in a future release(!) in the home assistant UI (including recurring events and a local calendar integration) we're now syncing events down to device and expanding them locally. This is instead of repeatedly polling the calendar data, we're now following best practices for efficient sync. However, this is a bug in the common library that handles the expansion logic.

allenporter commented 1 year ago

I've reproduced the bug in the underlying gcal_sync library https://github.com/allenporter/gcal_sync/pull/137 and will be pulling in a patch soon.

blair287 commented 1 year ago

Wow that was fast thanks a lot. Great work.

MagieMalone commented 1 year ago

Thanks a lot for all the work!

FL100 commented 1 year ago

Indeed - many thanks for a good job. B.t.w. I love the Google Agenda solution and use it for many things. I find it one of the most handy and amazing things in my HA setup.