franc6 / ics_calendar

Provides an ICS (icalendar) platform for the Home Assistant calendar
Apache License 2.0
154 stars 23 forks source link

Calender Recently Started Failing Due to "recurring_ical_events.InvalidCalendar: The event definition for [...] only contains modifications." #145

Open Ionshard opened 3 months ago

Ionshard commented 3 months ago

Recently my Google Calendar has disappeared from Home Assistant. When I checked the logs I saw many failures of:

2024-08-26 15:13:38.131 ERROR (SyncWorker_7) [custom_components.ics_calendar.calendar] update: Work: Failed to parse ICS!
Traceback (most recent call last):
  File "/config/custom_components/ics_calendar/calendar.py", line 293, in update
    self.event = self.parser.get_current_event(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/ics_calendar/parsers/parser_rie.py", line 117, in get_current_event
    for event in rie.of(self._calendar).between(
                 ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 1124, in of
    return CalendarQuery(
           ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 972, in __init__
    component_adapter.collect_components(calendar, self._skip_errors)
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 640, in collect_components
    result.append(Series(components))
                  ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 335, in __init__
    raise InvalidCalendar(
recurring_ical_events.InvalidCalendar: The event definition for 68bq8tk0lgptlose120drrk0sk@google.com only contains modifications.

Downloading the .ics file myself and looking for that event gave me this (obfuscated, since this is a work calendar)

BEGIN:VEVENT
DTSTART;TZID=America/Edmonton:20231004T130000
DTEND;TZID=America/Edmonton:20231004T160000
DTSTAMP:20240826T184819Z
ORGANIZER;CN=fake.name@business.com:mailto:fake.name@business.com
UID:68bq8tk0lgptlose120drrk0sk@google.com
ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=OPT-PARTICIPANT;PARTSTAT=ACCEPTED;CN=fake.name2@business.com;X-NUM-GUESTS=0:mailto:fake.name2@business.com
ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=ACCEPTED;CN=fake.name@business.com;X-NUM-GUESTS=0:mailto:fake.name@business.com
ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=OPT-PARTICIPANT;PARTSTAT=NEEDS-ACTION;CN=fake.name3@business.com;X-NUM-GUESTS=0:mailto:fake.name3@business.com
ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=OPT-PARTICIPANT;PARTSTAT=NEEDS-ACTION;CN=my.name@business.com;X-NUM-GUESTS=0:mailto:my.name@business.com
X-GOOGLE-CONFERENCE:https://meet.google.com/not-real-url
RECURRENCE-ID;TZID=America/Edmonton:20231004T130000
CREATED:20230914T155513Z
DESCRIPTION:Xxxx recently mentioned that watching people code is interestin
 g. I'm also going to commit to actually coding at this time. Join whenever\
 n\nJoin with Google Meet: https://meet.google.com/not-real-url\nOr dial: (C
 A) +1 647-734-3522 PIN: 123456789#\nMore phone numbers: https://tel.meet/no
 t-real-url?pin=1234567899999&hs=7\n\nLearn more about Meet at: https://supp
 ort.google.com/a/users/answer/9282720
LAST-MODIFIED:20240626T170317Z
SEQUENCE:1
STATUS:CONFIRMED
SUMMARY:Code Stream
TRANSP:OPAQUE
END:VEVENT

The thing is this event is very old and has been in my calendar for a long time, but it only stopped working recently. I can see from the stack trace this may be in the dependency recurring_ical_events but figured I'd raise this here to validate that before I go opening up tickets upstream.

Thanks Victor

daniel-narberhaus commented 3 months ago

Same issue on my side. Seems that it occurred first time just after the HACS 2.0 update.

Ionshard commented 3 months ago

Same issue on my side. Seems that it occurred first time just after the HACS 2.0 update.

To add to the data pool, I myself have not updated to HACS 2.0 yet.

franc6 commented 3 months ago

Thanks for the report. That does look like a problem in recurring_ical_events, but it may be one that I can work around. Note the entry shows it was created on 2023-09-14, but both last modified and dtstamp show 2024-08-26. I suspect the problem has something to do with that, but that’s odd to me. I might be able to remove or update the data since that info isn’t something that HA’s calendar entries know about, and is thus ignored. If you do report a problem upstream, please reply here with a link to the report so I know about it. IIRC, bug entries are best received with a full (but minimal) ics and example code. Mostly that means adding BEGIN:VCALENDAR/END:VCALENDAR to what you posted here. If I get the time to report the problem first, I’ll make note of it here, too.

jhclee commented 3 months ago

Adding a datapoint: I had this error but when I reverted back to Home Assistant Core 2024.8.2 it was fixed. I can also confirm that HACS 2.0.0 doesn't break it either.

Sorry I am not well versed enough to be able to provide a useful log, but hoping that this data point might help!

snap4ward commented 3 months ago

I rolled back to 2024.8.2 with no other changes and the problem was also solved for me.

r0bb10 commented 3 months ago
Logger: custom_components.ics_calendar.calendar
Source: custom_components/ics_calendar/calendar.py:266
integration: ics Calendar ([documentation](https://github.com/franc6/ics_calendar), [issues](https://github.com/franc6/ics_calendar/issues))
First occurred: 15:42:28 (5 occurrences)
Last logged: 15:42:45

async_get_events: Formula Uno: Failed to parse ICS!
Traceback (most recent call last):
  File "/config/custom_components/ics_calendar/calendar.py", line 266, in async_get_events
    event_list = self.parser.get_event_list(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/ics_calendar/parsers/parser_rie.py", line 67, in get_event_list
    for event in rie.of(self._calendar).between(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 1045, in between
    return self._between(start, stop)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 1058, in _between
    return self._occurrences_to_components(self._occurrences_between(start, end))
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 1065, in _occurrences_between
    occurrences.extend(series.between(start, end))
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 545, in between
    if occurrence.is_in_span(span_start, span_stop):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 883, in is_in_span
    return time_span_contains_event(span_start, span_stop, self.start, self.end)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 161, in time_span_contains_event
    raise PeriodEndBeforeStart(
recurring_ical_events.PeriodEndBeforeStart: the event must start before it ends(start: 2024-05-05 22:00:00+02:00 end: 2024-05-05 00:00:00+02:00)

Same here, does not work even with a new created calendar.

franc6 commented 3 months ago

It seems that the main issue is probably caused by a dependency of recurring_ical_events, as one of its dependencies is also a dependency of the caldav component of HA. An update to that could easily cause this when using the newer version of HA.

However, the problem that @r0bb10 sees is different. That looks like a direct problem in recurring_ical_events, and is probably unrelated to the other issues. There's no reason an event has to start before it ends, as setting the same time is common for all day events. When I get the chance, I'll report these problems upstream. Does going back to the older version of HA fix your problem, too, @r0bb10 ? If so, that suggests somebody added recurring_ical_events as a dependency of HA itself.

JonasDoebertin commented 3 months ago

I have the same issue currently:

Logger: custom_components.ics_calendar.calendar
Source: custom_components/ics_calendar/calendar.py:266
Integration: ics Calendar (Dokumentation, Probleme)
First occurred: 30. August 2024 at 08:59:59 (1631 Vorkommnisse)
Last logged: 07:58:25

async_get_events: Work Jonas: Failed to parse ICS!
Traceback (most recent call last):
  File "/config/custom_components/ics_calendar/calendar.py", line 266, in async_get_events
    event_list = self.parser.get_event_list(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/ics_calendar/parsers/parser_rie.py", line 67, in get_event_list
    for event in rie.of(self._calendar).between(
                 ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 1124, in of
    return CalendarQuery(
           ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 972, in __init__
    component_adapter.collect_components(calendar, self._skip_errors)
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 640, in collect_components
    result.append(Series(components))
                  ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 335, in __init__
    raise InvalidCalendar(
recurring_ical_events.InvalidCalendar: The event definition for ZOOM98319447321 only contains modifications.
iamjackg commented 2 months ago

recurring_ical_events did recently change code around that line: https://github.com/niccokunzmann/python-recurring-ical-events/commit/a658b1d7f200346f980e435c15e38946a9bc391e

They also released version 3.0.0 a couple weeks ago, which might have broken things since it's a major version bump.

https://github.com/mampfes/hacs_waste_collection_schedule/ also pulls it in as a dependency, but actually specifies a lower minimum version than this project, so it shouldn't be an issue.

AttilaPazmandi commented 2 months ago

Same issue for me as well. Today HACS has been updated to 2.0.1 but still the same.

Traceback (most recent call last): File "/config/custom_components/ics_calendar/calendar.py", line 266, in async_get_events event_list = self.parser.get_event_list( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/ics_calendar/parsers/parser_rie.py", line 67, in get_event_list for event in rie.of(self._calendar).between( ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 1124, in of return CalendarQuery( ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 972, in init component_adapter.collect_components(calendar, self._skip_errors) File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 640, in collect_components result.append(Series(components)) ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 335, in init raise InvalidCalendar( recurring_ical_events.InvalidCalendar: The event definition for 040000008200E00074C5B7101A82E0080000000010C47F73AFFBD60100000000000000001000000026C93D2AFAD562499E22423388A2FA79 only contains modifications.

All my calendars are shown as empty.

emtlovell commented 2 months ago

Also having this issue., But it works when reverting back to 2024.8.2

async_get_events: Trimble: Failed to parse ICS! Traceback (most recent call last): File "/config/custom_components/ics_calendar/calendar.py", line 266, in async_get_events event_list = self.parser.get_event_list( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/ics_calendar/parsers/parser_rie.py", line 67, in get_event_list for event in rie.of(self._calendar).between( ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 1124, in of return CalendarQuery( ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 972, in init component_adapter.collect_components(calendar, self._skip_errors) File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 640, in collect_components result.append(Series(components)) ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 335, in init raise InvalidCalendar( recurring_ical_events.InvalidCalendar: The event definition for 63p27l030b2hsb8sedpkv1r701@google.com only contains modifications.

delorean02 commented 2 months ago

Same thing with 2024.9.1. I'm rolling back to 2024.8.2.

AttilaPazmandi commented 2 months ago

This error has been fixed in the python-recurring-ical-events library. Hopefully the next version bump will fix this in HomeAssistant

franc6 commented 2 months ago

There was a new release of recurring-ical-events today, and I don't observe the problem with it. I also got the time today to finish off the other changes I've been very slowly working on, and I've made a release today. Please check with version 5.0.0, and report here if that fixes the problems for you. Please pay special attention to the release notes for this version.

Ionshard commented 2 months ago

I have updated to 5.0 but I am still seeing the issue. I have recently completely migrated my HA system to another device so I haven't been able to recreate this upstream yet.

r0bb10 commented 2 months ago

same issue also here, updated to 5.0 and reconfigured via ui.

url tested: https://www.formula1.it/agcalarm.ics


Logger: homeassistant.helpers.entity
Source: helpers/entity.py:942
First occurred: 06:33:30 (1 occurrences)
Last logged: 06:33:30

Update for calendar.formula_1 fails
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 942, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1302, in async_device_update
    await hass.async_add_executor_job(self.update)
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/ics_calendar/calendar.py", line 202, in update
    self.data.update()
  File "/usr/src/homeassistant/homeassistant/util/__init__.py", line 184, in wrapper
    result = method(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/ics_calendar/calendar.py", line 319, in update
    self.parser.set_content(self._calendar_data.get())
  File "/config/custom_components/ics_calendar/parsers/parser_ics.py", line 33, in set_content
    self._calendar = Calendar(re.sub(self._re_method, "", content))
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/ics/icalendar.py", line 69, in __init__
    self._populate(containers[0])  # Use first calendar
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/ics/component.py", line 59, in _populate
    parser(self, lines)  # Send a list or empty list
    ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/ics/parsers/icalendar_parser.py", line 71, in parse_vevent
    calendar.events = set(map(event_factory, lines))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/ics/parsers/icalendar_parser.py", line 69, in event_factory
    return Event._from_container(x, tz=calendar._timezones)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/ics/component.py", line 31, in _from_container
    k._populate(container)
  File "/usr/local/lib/python3.12/site-packages/ics/component.py", line 62, in _populate
    parser(self, lines[0])  # Send the element
    ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/ics/parsers/event_parser.py", line 26, in parse_dtstart
    event.begin = iso_to_arrow(line, tz_dict)
    ^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/ics/event.py", line 157, in begin
    raise ValueError('Begin must be before end')
ValueError: Begin must be before end
daniel-narberhaus commented 2 months ago

Generally, the new version works, as long "Include all day events" isn't checked.

@franc6 nice to have a configuration UI now, thanks! Just some small steps to go to get, maybe, for the all day events.

franc6 commented 2 months ago

Thanks for the updated reports. That's odd, since the unit tests cover most of these cases, and don't show problems parsing for the same date. I'm guessing there's something different with your setup than with mine, but I'm not sure what that could be yet, since installing 5.0.0 should have updated to the same dependency versions.

But please note for now, there's another problem with 5.0.0, and I recommend downgrading back to 4.2.0 for now. With 5.0.0, I'm now seeing calendars disappearing -- that didn't happen all day yesterday, but I won't have time to get back to this until next week.

Ionshard commented 2 months ago

I only use ics_calendar for my work calendar (which isn't working in either v4 or v5) so I can stay on v5 and beta test anything you need. My HA setup is back to "close enough to stable" for me to get back into the nitty gritty.

daniel-narberhaus commented 2 months ago

The new version 5.0.2 did it for me! Perfect! Thanks to all!

AttilaPazmandi commented 2 months ago

With 5.0.2 it works fine for me. Thank you!

image
twmcelroy commented 2 months ago

I'll throw in I have been having this same issue. I am on 5.0.2 and reconfigured via UI. I can get it to work if I switch the parser to ICS but then the times seem off and even the events aren't accurate. It is a work calendar that only shows if I'm busy or not. I usually have it ignore free events. Here is the error:

This error originated from a custom integration. Logger: custom_components.ics_calendar.calendar Source: custom_components/ics_calendar/calendar.py:321 integration: ICS Calendar (documentation, issues) First occurred: 3:36:19 PM (1 occurrences) Last logged: 3:36:19 PM update: Work Calendar: Failed to parse ICS! Traceback (most recent call last): File "/config/custom_components/ics_calendar/calendar.py", line 321, in update self.event = self.parser.get_current_event( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/config/custom_components/ics_calendar/parsers/parser_rie.py", line 118, in get_current_event for event in rie.of(self._calendar).between( ^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 1124, in of return CalendarQuery( ^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 972, in __init__ component_adapter.collect_components(calendar, self._skip_errors) File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 640, in collect_components result.append(Series(components)) ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 335, in __init__ raise InvalidCalendar( recurring_ical_events.InvalidCalendar: The event definition for 040000008200E00074C5B7101A82E008000000005C0A00BBF6ADD9010000000000000000100000009857DCEC4949524E84206355485148DE only contains modifications.

Here is one of the events BEGIN:VEVENT UID:040000008200E00074C5B7101A82E00800000000E082E21C10D5D901000000000000000 010000000AE80C854F8497C4DAE8A575AAD4D9DBC RECURRENCE-ID;TZID=Eastern Standard Time:20231020T153000 SUMMARY:Free DTSTART;TZID=Eastern Standard Time:20231020T153000 DTEND;TZID=Eastern Standard Time:20231020T180000 CLASS:PUBLIC PRIORITY:5 DTSTAMP:20240912T204017Z TRANSP:TRANSPARENT STATUS:CONFIRMED SEQUENCE:2 X-MICROSOFT-CDO-APPT-SEQUENCE:2 X-MICROSOFT-CDO-BUSYSTATUS:FREE X-MICROSOFT-CDO-INTENDEDSTATUS:BUSY X-MICROSOFT-CDO-ALLDAYEVENT:FALSE X-MICROSOFT-CDO-IMPORTANCE:1 X-MICROSOFT-CDO-INSTTYPE:3 X-MICROSOFT-DONOTFORWARDMEETING:FALSE X-MICROSOFT-DISALLOW-COUNTER:FALSE X-MICROSOFT-REQUESTEDATTENDANCEMODE:DEFAULT X-MICROSOFT-ISRESPONSEREQUESTED:FALSE END:VEVENT

Looks like it fails on the first event to fail. I removed the first one I found and then removed the calendar and added it back and found another event.

Ionshard commented 2 months ago

Updated to v5.0.2, completely new service configured via UI. I am also still getting the same error

Traceback (most recent call last):
  File "/config/custom_components/ics_calendar/calendar.py", line 321, in update
    self.event = self.parser.get_current_event(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/ics_calendar/parsers/parser_rie.py", line 118, in get_current_event
    for event in rie.of(self._calendar).between(
                 ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 1124, in of
    return CalendarQuery(
           ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 972, in __init__
    component_adapter.collect_components(calendar, self._skip_errors)
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 640, in collect_components
    result.append(Series(components))
                  ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/recurring_ical_events.py", line 335, in __init__
    raise InvalidCalendar(
recurring_ical_events.InvalidCalendar: The event definition for 68bq8tk0lgptlose120drrk0sk@google.com only contains modifications.
franc6 commented 2 months ago

I goofed earlier when updating the dependencies. They should be updated more properly now -- please install 5.0.3 and try again. If it still fails, those who know how, please report the versions of python, icalendar, and recurring_ical_events.

Thanks!

Ionshard commented 2 months ago

Just updated to v5.0.3 and my calendar works again!

Thank you for your hard work!

twmcelroy commented 2 months ago

So far so good, but I deleted the troublesome events. I'll keep an eye on it to make sure it stays operational.