caldav doesn't work with public calendars

[cmsj]

The problem

I have a public calendar shared from iCloud, but I can't get it to work with the caldav platform - the logs suggest the request is Unauthorized, but it shouldn't need to be authorized, and indeed I can wget the calendar URL without any login details.

I don't particularly want to give Home Assistant credentials for my iCloud account, and the calendar in question doesn't contain any personal information, so anonymous access seems appropriate. Am I doing something wrong with the configuration?

What version of Home Assistant Core has the issue?

2022.4.5

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Container

Integration causing the issue

caldav

Link to integration documentation on our website

https://www.home-assistant.io/integrations/caldav/

Diagnostics information

No response

Example YAML snippet

calendar:
  - platform: caldav
    url: https://p102-caldav.icloud.com/published/2/LOTSOFLETTERSANDNUMBERS

Anything in the logs that might be useful for us?

2022-04-17 21:25:40 ERROR (MainThread) [homeassistant.components.calendar] Error while setting up caldav platform for calendar
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 249, in _async_setup_platform
    await asyncio.shield(task)
  File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/caldav/calendar.py", line 87, in setup_platform
    calendars = client.principal().calendars()
  File "/usr/local/lib/python3.9/site-packages/caldav/davclient.py", line 350, in principal
    self._principal = Principal(client=self, *largs, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/caldav/objects.py", line 399, in __init__
    cup = self.get_property(dav.CurrentUserPrincipal())
  File "/usr/local/lib/python3.9/site-packages/caldav/objects.py", line 174, in get_property
    foo = self.get_properties([prop], **passthrough)
  File "/usr/local/lib/python3.9/site-packages/caldav/objects.py", line 197, in get_properties
    response = self._query_properties(props, depth)
  File "/usr/local/lib/python3.9/site-packages/caldav/objects.py", line 141, in _query_properties
    return self._query(root, depth)
  File "/usr/local/lib/python3.9/site-packages/caldav/objects.py", line 159, in _query
    ret = getattr(self.client, query_method)(
  File "/usr/local/lib/python3.9/site-packages/caldav/davclient.py", line 398, in propfind
    return self.request(url or self.url, "PROPFIND", props,
  File "/usr/local/lib/python3.9/site-packages/caldav/davclient.py", line 618, in request
    raise ex
caldav.lib.error.AuthorizationError: AuthorizationError at 'https://p102-caldav.icloud.com/published/2/LOTSOFLETTERSANDNUMBERS', reason 'Unauthorized'

Additional information

No response

[probot-home-assistant[bot]]

caldav documentation caldav source _{^{(message by IssueLinks)}}

[cmsj]

I wonder if this is ultimately because that's a published ICS file and not actually CalDav?

[PW999]

I have the same issue with an Outlook published calendar. It also seems to be a duplicate of #66599

The cause of the issue seems to be this line of code return self.request(url or self.url, "PROPFIND", props, {'Depth': str(depth)}) in the caldav client itself (https://github.com/python-caldav/caldav/blob/master/caldav/davclient.py#L392). This method seems to be using PROPFIND as the method for the request and most likely the servers only allow GET requests.

Not sure if there's a way around this

//edit: dug a bit deeper in some other issues here and it seems like there's a big release on it's way for the webdav client: https://github.com/home-assistant/core/issues/65804#issuecomment-1098551298 . Not sure if it's going to resolve this issue but I'd rather wait and see if it works with that branch prior to start digging in the code to see how I could get this working :)

[tobixen]

Try to use just https://caldav.icloud.com/ as the URL, and it should work.

There seems to be a regression somewhere that the caldav integration earlier worked also for ics feeds and not only caldav URLs. I'm not sure if this regression is in the python caldav library (which I'm maintaining) or if it's in the home assistant plugin, and I don't have capacity to do research on that now. I can only point out that there should be no need of using the caldav library at all when utilizing a ics feed :-)

[tobixen]

The python caldav library 0.9 has been released now, and it's expected to solve some issues - but probably not this one.

[cmsj]

@tobixen using https://caldav.icloud.com/ as the URL does work, but then I have to provide auth details for my iCloud account, and that's what I'm ideally trying to avoid. I'm afraid I know very little about caldav, so I have no idea how the library would distinguish between a caldav URL that needs authentication vs a published ICS feed, and if the answer is "stop using caldav if you don't actually want to do caldav" then that's fine :)

[tobixen]

I think that if the HomeAssistant module was working with ICS feeds earlier, then I think we ought to look into that and get it to work again - but I don't have the capacity today at least, and I think the problem is in the HA-module (which I haven't looked much into), not with the python caldav library (which I'm taking some responsibility for) :-)

For the clarification - caldav is a two-way standard for communicating with a calendar server (retrieve events, search a calendar for some specific date range, get a list of updated events, save changes, add events, etc). All calendar data passed will be in the ICS (icalendar) format. Most modern calendar servers also allows an ICS feed to be published (for calendars that are public), basically allowing a client to get a complete copy of the full calendar, but this is not a part of the caldav standard, and should be accessible even without using the caldav library.

[PW999]

For me it's the first time I'm trying to use the component, so I can not say if there's any form of regression.

Tbh, when checking out the Caldav code it's indeed pretty clear that it's not made at all for just reading ICS files. I think I'll have to look into custom components like e.g. https://github.com/franc6/ics_calendar even though at first sight it seems to be missing the handy "customer_calendar" feature which the caldav integration does support.

[tobixen]

Tbh, when checking out the Caldav code it's indeed pretty clear that it's not made at all for just reading ICS files.

The python caldav library or the home assistant module?

I haven't looked into the home assistant module yet, but I think it would make sense to have one calendar module for home assistant that would accept both caldav and an ics feed.

[github-actions[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[tobixen]

This issue is certainly not fixed.

From my perspective it's not a bug per se, as a "caldav" module should not need to support a plain ics feed. Reporters says that it was working for them before (2021-11?), so from that it seems to be a regression issue.

[allenporter]

Feature requests are discussed in the community forums and not tracked as issues. This doesn't look like a regression but misunderstanding of the caldav component features.

[tobixen]

I don't have any beef in this personally, I would just like to point out yet another time that this may be a regression issue. I think that regression issues belongs in the issue tracker and generally they ought to be fixed.

If I'm not mistaken, two persons have reported that it was working earlier.

I do find it a bit strange though. At least from the caldav python library side, I would believe also the old version would fail trying to do a PROPFIND request towards the ics feed, so unless something has changed in the HA-module I would tend to believe that those reports are due to a simple misunderstanding.

The simple fix - if this issue is to be fixed - could possibly be to first try a GET towards the URL and just eat the results if valid ical data is returned.

[allenporter]

Yes, if it's a regression is should be here. But i don't see why it is one, except for confusion about what the caldav library is supposed to do, as caldav has never supported ics file feeds, right? It's only using the caldav library.

[tobixen]

It is.

[PW999]

For me it never worked before, I was trying to setup this integration for the first time and I was kind of hoping that it would work with the anonymous links. Digging a bit deeper in the caldav code to figure out why it didn't work made it clear that even though they offer it as "caldav", it's nothing even close to being caldav.

But yeah, it would be cool to have an HA integration that support ICS files and offers the same kind of neat functionalities.

[allenporter]

FWIW I have a "local calendar" integration in the works https://github.com/allenporter/home-assistant-core/tree/local-calendar/homeassistant/components/local_calendar that can read ICS files that i'm working on getting ready for core.

I'm not sure that it's going to be able to sync with remote calendar calendar, but maybe a one-time import...

[allenporter]

FWIW I have a "local calendar" integration in progress https://github.com/allenporter/home-assistant-core/tree/local-calendar/homeassistant/components/local_calendar that can read ICS files that i'm working on getting ready for core. I'm not sure that it's going to be able to sync with remote calendar calendar, but maybe a one-time import, or maybe that could be a custom component.