Parser error in X-APPLE-STRUCTURED-EVENT field

[fpose]

Hello,

the date_search() method issues a parser error, if an event contains a X-APPLE-STRUCTURED-LOCATION record. I think it has something to do with the double-backslash-escaped newlines. Other caldav clients do not complain about it. Do you think it is a icalendar syntax problem? Could there be a workaround?

Thank you very much, Florian

    ...
    results = calendar.date_search(start, end)
  File "/usr/lib/python3.4/site-packages/caldav/objects.py", line 510, in date_search
    Event(self.client, url=self.url.join(r), data=results[r][cdav.CalendarData.tag], parent=self))
  File "/usr/lib/python3.4/site-packages/caldav/objects.py", line 724, in __init__
    self.data = data
  File "/usr/lib/python3.4/site-packages/caldav/objects.py", line 801, in _set_data
    self._instance = vobject.readOne(to_unicode(self._data))
  File "/usr/lib/python3.4/site-packages/vobject/base.py", line 1156, in readOne
    allowQP))
  File "/usr/lib/python3.4/site-packages/vobject/base.py", line 1101, in readComponents
    vline = textLineToContentLine(line, n)
  File "/usr/lib/python3.4/site-packages/vobject/base.py", line 925, in textLineToContentLine
    return ContentLine(*parseLine(text, n), **{'encoded': True,
  File "/usr/lib/python3.4/site-packages/vobject/base.py", line 813, in parseLine
    raise ParseError("Failed to parse line: {0!s}".format(line), lineNumber)
vobject.base.ParseError: At line 15: Failed to parse line: X-APPLE-STRUCTURED-LOCATION;VALUE=URI;X-APPLE-MAPKIT-HANDLE=CAESvAEaEgmX5esy/OVJQBGXkXpP5aQYQCJiCgtEZXV0c2NobGFuZBICREUaE05vcmRyaGVpbi1XZXN0ZmFsZW4qBUtsZXZlMgVLbGV2ZToFNDc1MzNSDkp1bmdmZXJuZ3JhYmVuWgIyNWIRSnVuZ2Zlcm5ncmFiZW4gMjUqEUp1bmdmZXJuZ3JhYmVuIDI1MhFKdW5nZmVybmdyYWJlbiAyNTILNDc1MzMgS2xldmUyC0RldXRzY2hsYW5kODlAAA==;X-APPLE-RADIUS=70.58738448026273;X-APPLE-REFERENCEFRAME=1;X-TITLE=Löschzug Kellen\

Here is the event that fails:

BEGIN:VEVENT
CREATED:20190103T070318Z
DTEND;TZID=Europe/Berlin:20190117T230000
DTSTAMP:20190103T070319Z
DTSTART;TZID=Europe/Berlin:20190117T180000
LAST-MODIFIED:20190103T070318Z
LOCATION:Löschzug Kellen\nJungferngraben 25\n47533 Kleve\nDeutschland
RRULE:FREQ=WEEKLY;INTERVAL=2;UNTIL=20191231T225959Z;BYDAY=TH;WKST=SU
SEQUENCE:0
SUMMARY:Arbeitsgruppe
TRANSP:OPAQUE
UID:B4D490C2-1261-471A-93E8-7D27FB5DA3C8
URL;VALUE=URI:
X-APPLE-STRUCTURED-LOCATION;VALUE=URI;X-APPLE-MAPKIT-HANDLE=CAESvAEaEgmX      
 5esy/OVJQBGXkXpP5aQYQCJiCgtEZXV0c2NobGFuZBICREUaE05vcmRyaGVpbi1XZXN0ZmFs     
 ZW4qBUtsZXZlMgVLbGV2ZToFNDc1MzNSDkp1bmdmZXJuZ3JhYmVuWgIyNWIRSnVuZ2Zlcm5n     
 cmFiZW4gMjUqEUp1bmdmZXJuZ3JhYmVuIDI1MhFKdW5nZmVybmdyYWJlbiAyNTILNDc1MzMg     
 S2xldmUyC0RldXRzY2hsYW5kODlAAA==;X-APPLE-RADIUS=70.58738448026273;X-APPL     
 E-REFERENCEFRAME=1;X-TITLE=Löschzug Kellen\\nJungferngraben 25\\n47533 K     
 leve\\nDeutschland:geo:51.796759,6.161031
X-APPLE-TRAVEL-ADVISORY-BEHAVIOR:AUTOMATIC
BEGIN:VALARM
ACTION:NONE
TRIGGER;VALUE=DATE-TIME:19760401T005545Z
END:VALARM
END:VEVENT

[tobixen]

Sorry for the slow response on this.

I haven't checked if extra fields starting with X is allowed or not in the standard, but I would like this to pass.

The traceback is from the vobject module, so it's possible to raise the issue there. Check if you're running the latest release of vobject.

That said, I will look into why the object is instantiated here. I'm on the mobile now, but if there is no good reason to instantiate the object it should not be done.

Another thing, there has been talks on ditching vobject for a long time. There is even two pull requests written for this purpose. Only reason why we stick to vobject is that said pull requests were not backwards compatible.

[tobixen]

See also my comments on #36

[tobixen]

Fields starting with X should be allowed, but semicolons on the left side of the equality sign is probably not allowed in the standard, hence this is a bug from the client injecting the event and/or the server side.

That said, with #36 solved now (so far only in the dev branch, but the 0.6.0 release will be out soon) hopefully it should be possible to search for those events without triggering the problem. Also, I will accept pull requests if you'd like to try to make a workaround in the icalendar-fixup function.

Closing this issue as for now.

[tobixen]

Reopening this one to do more research.

[tobixen]

Possibly relevant ... https://github.com/sabre-io/vobject/issues/402 ... some ical data with an invalid line break

[tobixen]

I've reproduced the problem:

>>> vobject.readOne(ics)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/dist-packages/vobject/base.py", line 1155, in readOne
    return next(readComponents(stream, validate, transform, ignoreUnreadable,
  File "/usr/local/lib/python3.9/dist-packages/vobject/base.py", line 1101, in readComponents
    vline = textLineToContentLine(line, n)
  File "/usr/local/lib/python3.9/dist-packages/vobject/base.py", line 925, in textLineToContentLine
    return ContentLine(*parseLine(text, n), **{'encoded': True,
  File "/usr/local/lib/python3.9/dist-packages/vobject/base.py", line 813, in parseLine
    raise ParseError("Failed to parse line: {0!s}".format(line), lineNumber)
vobject.base.ParseError: At line 15: Failed to parse line: Jungferngraben 25 

The next is to check weather the ics is valid or not. If the ics is valid, then this is an issue that has to be addressed at the vobject project. If the ics is invalid (or if it proves very hard to get vobject fixed), then it's within the scope to make a workaround in the caldav library.

[tobixen]

The \\n in the ics presented is not a problem, ics treats it as the literal string \n, and treated like a line break by whatever application is using the X-APPLE-STRUCTURED-LOCATION-attribute.

The problem here is that the ics presented has trailing whitespace on the X-APPLE-STRUCTURED-LOCATION-attribute. I think it's quite obvious that it shouldn't be there, and I suppose it's also a breach of the the ical standard as both vobject fails (with a traceback) and also the icalendar library fails (by simply ignoring that attribute silently). This ought to be investigated more, but as for now I'm simply stripping away the extra white spaces. It also ought to be tested if this really fixes the problem for the original reporter, but I'm not really expecting any additional feedback after two and a half years ... and there also seems to be other issues, like indentation broken in the middle of the attribute, ref the linked sabre-issue.

[fpose]

Hello tobixen and thanks for getting back to this issue. I will try your fix and let you know! Regards, Florian

[zsamiatt]

Hello, have a problem with new home assitant caldav. I using early but I need using now. But can not working.

Logger: homeassistant.components.calendar Source: components/caldav/calendar.py:203 Integration: Naptár (documentation, issues) First occurred: 17:54:32 (1 occurrences) Last logged: 17:54:32

caldav: Error on device update! Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 446, in _async_add_entity await entity.async_device_update(warning=False) File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 702, in async_device_update await task File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, *self.kwargs) File "/usr/src/homeassistant/homeassistant/components/caldav/calendar.py", line 144, in update self.data.update() File "/usr/src/homeassistant/homeassistant/util/init.py", line 192, in wrapper result = method(args, **kwargs) File "/usr/src/homeassistant/homeassistant/components/caldav/calendar.py", line 203, in update results = self.calendar.date_search(start_of_today, start_of_tomorrow) File "/usr/local/lib/python3.10/site-packages/caldav/objects.py", line 750, in date_search objects = self.search(root, comp_class) File "/usr/local/lib/python3.10/site-packages/caldav/objects.py", line 812, in search (response, objects) = self._request_report_build_resultlist(xml, comp_class) File "/usr/local/lib/python3.10/site-packages/caldav/objects.py", line 779, in _request_report_build_resultlist response = self._query(xml, 1, 'report') File "/usr/local/lib/python3.10/site-packages/caldav/objects.py", line 167, in _query raise error.exception_by_methodquery_method caldav.lib.error.ReportError: ReportError at '500 Internal Server Error

b'<?xml version="1.0" encoding="utf-8"?>\n\n 4.3.1</s:sabredav-version>\n Sabre\VObject\ParseException</s:exception>\n This parser only supports VCARD and VCALENDAR files</s:message>\n</d:error>\n'', reason no reason

[tobixen]

File "/usr/local/lib/python3.10/site-packages/caldav/objects.py", line 167, in _query raise error.exception_by_methodquery_method caldav.lib.error.ReportError: ReportError at '500 Internal Server Error

b'\n\n](http://sabredav.org/ns%22%3E%5Cn) 4.3.1</s:sabredav-version>\n Sabre\VObject\ParseException</s:exception>\n This parser only supports VCARD and VCALENDAR files</s:message>\n</d:error>\n'', reason no reason

This is weird ... perhaps the calendar itself has become corrupted? Can you try to set up an empty calendar and see if the problem persists?

[zsamiatt]

I try new calendar. The calendar working good from MS outlook with caldav addon. But HA caldav get error.