search

pimutils / vdirsyncer

📇 Synchronize calendars and contacts.
https://vdirsyncer.pimutils.org/
Other
1.57k stars 162 forks source link

empty lines in events causes upload failures #1026

Open c-schwamborn opened 1 year ago

c-schwamborn commented 1 year ago

I guess the cause of this issue is a preexisting error on the source caldav servers calendar data assumably done by some faulty client implementation. But it annoyed the hell out of me as about 2 or 3 percent of the events failed to transfer between the two servers during a migration from a davical system to a nextcloud in a number of user calendars - so I started investigating.

It was always the same error: 415 'Unsupported Media Type'

Digging through some example events revealed an error in the SUMMARY attribute. Those where multiline entries with blank lines in it and each of the faulty ones contained a complete empty line, not just a line with a leading space followed by a newline. Removing those empty lines solved the issue with those events, but to handle the whole migration without having to handle each error by hand would either mean to dump the whole thing to files, fix the broken ones and upload the bunch to the new system, or modifying the vdirsyncer code - I decided to try the latter.

Before doing a pull request i would like to discuss the matter here, if this special case is worth changing the code and if what i did might have some unforeseen side effects. As I understand the RFC, neither ics nor vcard objects should contain blank lines, at least not those without at least a space indicating that its part of the previous attribute. So my guess is, that the item import could securely remove those lines from the raw data. At least in my tests that seemed to work.

As a reference, I replaced in vobject.py class Item: the following in the __init__ function:

- self._raw = raw
+ self._raw = "".join([l for l in raw.strip().splitlines(True) if l.strip('\r\n') ])
WhyNotHugo commented 1 year ago

I'd be okay with removing superfluous empty lines which are invalid as per the iCalendar spec.

Generally, we avoid tampering with the contents of items, but this seems a clear enough scenario. Do keep in mind that:

Content lines are delimited by a line break, which is a CRLF sequence (CR character followed by LF character).

From https://www.rfc-editor.org/rfc/rfc5545#section-3.1

WhyNotHugo commented 1 year ago

The code above makes sense as a proof-of-concept, though it's not the most efficient way to do it (you're stripping each line twice).