caldav + homeassistant + gmx -> "path handling problem"

[tobixen]

I got an email about problems with the German provider www.gmx.de. I will look more into it as soon as I get time.

[tobixen]

The details from the error message looks like this (replaced some username with USERNAME and some uuid with UUID):

Path expected: /begenda/dav/username@gmx.de/<mailto:/begenda/dav/USERNAME@gmx.de/>
path found: /begenda/dav/UUID/

[tobixen]

Not 100% related to the reported problem, but still ... when sending a REPORT with this body, I get a 500 internal server error:

<?xml version="1.0" encoding="utf-8"?>
<C:calendar-query xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
  <D:prop>
    <C:calendar-data/>
  </D:prop>
  <C:filter>
    <C:comp-filter name="VCALENDAR"/>
  </C:filter>
</C:calendar-query>

While this one works:

<?xml version="1.0" encoding="utf-8"?>
<C:calendar-query xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
  <D:prop>
    <D:getetag/>
    <C:calendar-data/>
  </D:prop>
  <C:filter>
    <C:comp-filter name="VCALENDAR"/>
  </C:filter>
</C:calendar-query>

The difference here is that in the second example I ask for a calendar-data and getetag, while I'm only asking for calendar-data in the first one. Hence, it seems to be required to ask for getetag.

I'm a bit at a loss on how to fix this one. The quick fix would be to ensure that getetag is included in all REPORT queries. All until I find some other calendar server that gives a 500 internal server error if someone asks it for an getetag ... hm.

[tobixen]

Current test breakages looks like this:

============================================================================================================================ short test summary info ============================================================================================================================
FAILED tests/test_caldav.py::TestForServer_caldav_gmx_net::testCreateEvent - caldav.lib.error.AuthorizationError: AuthorizationError at 'https://caldav.gmx.net/begenda/dav/e497e168-9979-41d9-8138-82cc8699c8a2/pythoncaldav-test/20010712T182145Z-123401%40example.com.ics', reason Forbidden
FAILED tests/test_caldav.py::TestForServer_caldav_gmx_net::testObjectBySyncToken - caldav.lib.error.ReportError: ReportError at '422 Unprocessable Entity
FAILED tests/test_caldav.py::TestForServer_caldav_gmx_net::testSync - caldav.lib.error.ReportError: ReportError at '422 Unprocessable Entity
FAILED tests/test_caldav.py::TestForServer_caldav_gmx_net::testSearchEvent - assert 0 == 1
FAILED tests/test_caldav.py::TestForServer_caldav_gmx_net::testTodoDatesearch - RecursionError: maximum recursion depth exceeded
FAILED tests/test_caldav.py::TestForServer_caldav_gmx_net::testRecurringDateSearch - RecursionError: maximum recursion depth exceeded
============================================================================================================= 6 failed, 70 passed, 80 skipped in 2114.80s (0:35:14) =============================================================================================================

So sync-tokens are obviously not supported, most of the other tests (recursive search etc) are known to break in different ways on different server implementations. I think none of this matters for HomeAssistant.

[tobixen]

Sorry that I haven't had time to follow up on this. It's not forgotten, just want to get my plann project good enough to manage my task lists before catching up with this and other issues ...

[tobixen]

Right, I made a branch for working more with this issue, and made one commit that should fix the worst problems - and then I got busy with other things and forgot about it. I won't have time for working more towards this server before releasing 1.3, but at least the worst problems should be fixed in 1.3.

[paulcombuse]

Thanks, @tobixen - if I could assist you in any way, please let me know. I also played arround with different parameters, but still no success for me. Looking forward to the 1.3 release

[paulcombuse]

Probably you know this already: regarding the etag and the ctag I found quite a good explanation on this website https://sabre.io/dav/building-a-caldav-client/ - basically saying these tags are kind of version or change tags for the whole calendar (ctag) or only for an entity (etag), meaning that if the ctag changes, something in the calendar has changed and if the etag of an entity changes this indicates that some parts of the element have changed. And for sure you know this document: https://datatracker.ietf.org/doc/html/rfc4918

[tobixen]

1.3 was released long ago. I should get either 1.4 or 1.3.7 out soon'ish.

As far as I've understood, ctag/etag was partially obsoleted for sync-tokens. There is no specific support for etag or ctag in the caldav library, but it does support sync-tokens.

According to my discovery further up, the server would return a "500 internal server error" if a report request is sent without the getetag property. Now this is an error at the server side, for two reasons. Firstly, sending a report request without the getetag property is fully legit and according to the standards, so it should return data. Secondly, if the client behaves in ways that the server thinks is non-standard, it should yield a 4xx-error rather than a 5xx-error to show that the problem is at the client side.

Adding some code to ensure that getetag is always included in report request is sort of trivial, but I won't want to do it, because it may break things for other users. Perhaps there are other calendar servers out there that works completely fine without the getetag request, but doesn't support etags and yields a 500 internal server error when I'm asking for etags.

I do have another thought on how to solve this (and many other problems) - by passing a list of "compability tweaks" in the DAVClient constructor. I will for sure not have time for it this week.

One problem for me is that I don't have a test account towards gmx.de. It's a paid service. Last time I worked around it by creating an account and choosing some option that I'd pay "cash by mail", though it feels dishonest. Paying for being able to test the library towards gmx and create compability tweaks for it also feels wrong. If you (or someone) could contact them and convince them to give me a free testing account (which will only be used for testing the caldav library, and that without taxing their servers too much) then there is a bigger chance that I will prioritize this issue :-)

[paulcombuse]

@tobixen: Thanks for the update and the clarification of what is going on/wrong. Regarding GMX I set up a test account for you month ago, but for sure there were too many other things, so I will forward my email to you again - just in case you have another idea how to get this issue working (though I understood the trouble happens on the server side). All the best!

[paulcombuse]

Erm, @tobixen could you tell me where version 1.3.x or higher is available? The release page still shows 1.2.1 from March as the latest release https://github.com/python-caldav/caldav/releases

[tobixen]

https://pypi.org/project/caldav/ - shows 1.3.6

I'm sorry wrg of the test account, I simply remember wrong (possibly mixing it up with another service). I can confirm that I do have a gmx account in my private test config.

[tobixen]

I always thought the "github release"-thing was a pretty moot feature, but apparently I was mistaken on that. I'll try to remember it when I release 1.3.7 and/or 1.4.0. In any case, commands like sudo pip install -U caldav is supposed to give you the last version :-)