Help debugging a calendar query error

elliottjohnson commented 3 years ago

Hi,

I've been working with HomeAssistant (Core 2020.12.1 installed on a recent Gentoo server) in connecting to a Baïkal calendar and have been tracing an odd issue. In debugging I'm seeing that the query is handled by sabre-io/dav and then passed to sabre-io/vobject, but I believe sabre-io/dav is the correct place to discuss this issue. See below for further details.

I've connected successfully to the test calendar using AgenDav and Moz Thunderbird, so everything appears to be working well with Baïkal (v0.7.2). When HomeAssistant attempts to connect to the server, it sends several successful request (PROPFINDs with 207 results), but eventually uses python's caldav package (v0.6.1) to create the following calendar query to the Baïkal server.

<?xml version='1.0' encoding='utf-8'?>
<C:calendar-query xmlns:D="DAV" xmlns:C="urn:ietf:params:xml:ns:caldav" xmlns:I="http://apple.com/ns/ical/">
  <ns0:prop xmlns:ns0="DAV:">
    <C:calendar-data>
      <C:expand start="20210108T080000Z" end="20210115T080000Z"/>
    </C:calendar-data>
  </ns0:prop>
  <C:filter>
    <C:comp-filter name="VCALENDAR">
      <C:comp-filter name="VEVENT">
        <C:time-range start="20210108T080000Z" end="20210115T080000Z"/>
      </C:comp-filter>
    </C:comp-filter>
  </C:filter>
</C:calendar-query>

This query is received by the Baïkal instance running on apache (on the same host) and apache unfortunately returns a 500 error:

<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:sabredav-version>4.1.2</s:sabredav-version>
  <s:exception>Sabre\\VObject\\ParseException</s:exception>
  <s:message>This parser only supports VCARD and VCALENDAR files</s:message>
</d:error>

I put a print statement in saber/vobject's MimeDir.php file at the above error output and a timezone string is output, so I changed the timezone setting on the Baïkal calendar being queried and that change was reflected in the debugging output. I put a stack trace into that point in the code (baikal/vendor/sabre/vobject/lib/Parser/MimeDir.php after the default in the switch statement) and got this:

I'm not a rfc4791 expert, just a guy trying to get a shared calendar resource setup on his home server for use in some home automation. If anyone can look over the calendar-query or the backtrace to ensure it's valid or other acceptable caldav requests, please let me know.

Any suggestions on how to further debug this would be appreciated.

Thanks! Elliott

elliottjohnson commented 3 years ago

Looking closer at CalDAV\Plugin.php:513, I can see the call to VObject\Reader::read($tzResult[$tzProp]), which is where the timezone is being passed to vobject's parser. Apparently this has been discussed before https://github.com/sabre-io/Baikal/issues/695#issuecomment-394112977 and the following was suggested by @evert https://github.com/sabre-io/Baikal/issues/695#issuecomment-487299181

I have been able to get things to work if I perform the following modification to CalDAV\Plugin.php:

    public function calendarQueryReport($report)
    {
        $path = $this->server->getRequestUri();

        $needsJson = 'application/calendar+json' === $report->contentType;

        $node = $this->server->tree->getNodeForPath($this->server->getRequestUri());
        $depth = $this->server->getHTTPDepth(0);

        // The default result is an empty array
        $result = [];

        $calendarTimeZone = null;
        if ($report->expand) {
            // We're expanding, and for that we need to figure out the
            // calendar's timezone.
            $tzProp = '{'.self::NS_CALDAV.'}calendar-timezone';
            $tzResult = $this->server->getProperties($path, [$tzProp]);
            if (isset($tzResult[$tzProp])) {
                $calendarTimeZone = new DateTimeZone($tzResult[$tzProp]);
            } else {
                // Defaulting to UTC.
                $calendarTimeZone = new DateTimeZone('UTC');
            }
        }

I don't know what kind of effect doing so will have on other systems, but this change is working for me now. I'll do some more testing and report back if I have any further issues.

TheDarthFather commented 3 years ago

Hello, Seems I have the same problem with HA and the Baikal CalDav docker (This parser only supports VCARD and VCALENDAR files).

In HA log:

[homeassistant.components.calendar] caldav: Error on device update!
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 359, in _async_add_entity
    await entity.async_device_update(warning=False)
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 466, in async_device_update
    await task
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/src/homeassistant/homeassistant/components/caldav/calendar.py", line 146, in update
    self.data.update()
  File "/usr/src/homeassistant/homeassistant/util/__init__.py", line 239, in wrapper
    result = method(*args, **kwargs)
  File "/usr/src/homeassistant/homeassistant/components/caldav/calendar.py", line 203, in update
    results = self.calendar.date_search(start_of_today, start_of_tomorrow)
  File "/usr/local/lib/python3.8/site-packages/caldav/objects.py", line 594, in date_search
    response = self._query(root, 1, 'report')
  File "/usr/local/lib/python3.8/site-packages/caldav/objects.py", line 167, in _query
    raise error.exception_by_method[query_method](errmsg(ret))
caldav.lib.error.ReportError: 500 Internal Server Error

b'<?xml version="1.0" encoding="utf-8"?>\n<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">\n  <s:sabredav-version>4.1.2</s:sabredav-version>\n  <s:exception>Sabre\\VObject\\ParseException</s:exception>\n  <s:message>This parser only supports VCARD and VCALENDAR files</s:message>\n</d:error>\n'

I am not sure how I can apply the fix above - my docker exposed folders do not contain a Plugin.php ... any advice ?

elliottjohnson commented 3 years ago

I am not sure how I can apply the fix above - my docker exposed folders do not contain a Plugin.php ... any advice ?

Ah. To implement the fix I applied, you'll need to edit a file on the docker instance. I'm not well versed with docker to know what the recommended method to do this would be, but what I would do is to use docker exec to login to the instance, find where they installed sabre-io's CalDAV code (mine is in the web server directory).

From there open the Plugin.php file with a text editor (hopefully one that you know is installed), find the calendarQueryReport function in there, edit it to match what I have above and that should be it... simple, huh! :D

Hopefully that is a persistent change and I'd assume so, but again I've never used docker.

elliottjohnson commented 3 years ago

@TheDarthFather did any of that help you out?

TheDarthFather commented 3 years ago

Hi, Sorry for my late reply. Indeed that should be doable ... I realised later (docker noob here) that I can login to the instance and access all the files. PS: Unfortunately docker does not have MC and to find that file and edit that is going to be a pain :).

hdwinkel commented 3 years ago

Hi, the patch https://github.com/sabre-io/dav/issues/1318#issuecomment-757380175 worked for me. the file in the docker container can be copied to a local and back after the patch: docker cp baikal:/var/www/baikal/vendor/sabre/dav/lib/CalDAV/Plugin.php .

gador commented 3 years ago

Does it break anything else? If not, maybe it is worth creating a pull request for your change @elliottjohnson ?

Your patch helped me, too. Baikal is now working as expected with Home Assistant.

ykktrcb7 commented 2 years ago

is there an update for this issue?

doodieb15 commented 2 years ago

This problem is back with HA 2022.2.2. The "hack" to modify the Plugin.php script does not work anymore.

gador commented 2 years ago

Currently ongoing investigation of the issue here

doodieb15 commented 2 years ago

OK there is a fix in homeassitant:

https://github.com/python-caldav/caldav/pull/178/commits/93afdca23ce6048006f8aa17a3faa2c718db3bc5

With this and the fix above, everything works again as expected

screenagerbe commented 2 years ago

Been struggling with this for several weeks... with update to python-caldav v0.9.0 (see https://github.com/home-assistant/core/issues/65804 ) and this fix I've got everything running again in HA 2022.5.1

MrAlucardDante commented 2 years ago

This fix worked for me too, unfortunately, when running baikal in a docker container, we have to reapply this fix everytime the container is recreated. Will it be included into Baikal in a future release ?

screenagerbe commented 2 years ago

I had to reapply the fix too when I manually updated my Baikal server (running on a shared hosting service) to v0.9.1 and v0.9.2. Don't think this fix will be applied any time soon as only HA users are complaining about the issue.

mikedrawback commented 1 year ago

Is there any chance this will get fixed? I'm trying to connect my baikal calendars to Home Assistant and believe I have run into this issue

MrAlucardDante commented 8 months ago

If you use the docker image from ckulka, I have created a docker image that's based on his nginx variant (I don't use the apache/httpd variant, but since it's just the webserver, migrating should work out of the box) which includes this fix.

Here is the image name and tag : ghcr.io/mralucarddante/baikal-docker-hass:0.9.4-msmtp

And the code is on GitHub if you want to make sure I don't do anything shady in the Dockerfile (of course I don't) https://github.com/MrAlucardDante/baikal-docker-hass

evert commented 6 months ago

Few years late but this appeared in my notifications. It looks like that timezone field somehow got a timezone string, but this was never correct. The field should contain a full iCalendar object with a VTIMEZONE.

I wonder how the wrong data ever could have made it in the database!

apollo-sturdy commented 6 months ago

Few years late but this appeared in my notifications. It looks like that timezone field somehow got a timezone string, but this was never correct. The field should contain a full iCalendar object with a VTIMEZONE.

I wonder how the wrong data ever could have made it in the database!

It seems that Calendars created by Baikal set the timezone field to a timezone string! I opened an issue for it here: https://github.com/sabre-io/Baikal/issues/1241

derjoerg commented 6 months ago

@MrAlucardDante Thanks a lot for this docker image. Works like a charme.

justadri commented 5 months ago

Confirming that the modified Baikal Docker image from @MrAlucardDante works great (thank you!), however the "0.9.4-msmtp" tag no longer exists. I just used the "latest" tag and it worked fine; if you want to be version-specific, the current tag is just "0.9.4".

MrAlucardDante commented 5 months ago

Confirming that the modified Baikal Docker image from @MrAlucardDante works great (thank you!), however the "0.9.4-msmtp" tag no longer exists. I just used the "latest" tag and it worked fine; if you want to be version-specific, the current tag is just "0.9.4".

Correct. In order to automate the release of new versions, I use Renovate that update the dockerfile when ckulka releases a new version, and I use commitizen to automatically tag a new version, which is based on semver.

That is why there is no -msmtp variant. Also since the msmtp feature was included in the main release anyway, it makes sense to me to remove this suffix.

Anyway, enough of the technical stuff, thanks for the update.

sabre-io / dav

Help debugging a calendar query error #1318