Better CalDAV sync compatibility for calendar (Google calendar exported data and Apple iCal)

[mhaller]

Expected behaviour

Importing ICS calendar data exported from Google calendar in owncloud works. The syncing with CalDAV clients should work as well.

Actual behaviour

However, Apple iCal crashes while syncing with .../owncloud/remote.php/caldav/principals/USERNAME/. Inspection of ICS file showed that the exported data from Google calendar uses various formats for the UIDs. I assigned a new UID to each VEVENT. I also removed all alarms. Syncing between iCal and owncloud works now perfectly.

To assign new UIDs, the icalendar Python module was quite helpful. https://pypi.python.org/pypi/icalendar

Feature request

Owncloud could do this reassignment of new UIDs automatically while importing the data from the ICS file. This improves the ease of use for switching from Google to OC. Especially beginners will be thankful if import and sync with widely-used WebDAV-client iCal just works.

Steps to reproduce

1. Export Google calendar data as ICS file
2. Upload ICS file to OC
3. Import and create a new calendar in OC
4. Add an OC-WebDAV account in Apple iCal
5. Sync starts automatically
6. See in Console.app that iCal crashed while syncing
7. Delete old calendar
8. Fix ICS data: Reassign a new UID to each VEVENT, delete alarms
9. Uploaded fixed ICS file
10. Import and create a new calendar in OC
11. Add an OC-WebDAV account in Apple iCal
12. Sync starts automatically
13. Sync finished successfully

    Server configuration

Operating system: Linux 2.6.33.2 Web server: Apache/2.2.14 Database: MySQL 5.1.36 PHP version: 5.3.14 ownCloud version: 5.0.0

Client configuration

Browser: --- Operating system: Mac OS X 10.7.5 Apple iCal: 5.0.3

[tanghus]

Eh, shouldn't this issue be filed at Google and Apple then?

[mhaller]

Yeah, Apple could have done better in terms of robustness of dealing with iCalendar (aka RFC 5545) and Google could have done better with generating working UIDs. But, maybe Apple wants to be compatible only to OS X Server and maybe Google has no interest in exporting data which can be used to switch easily to another calendar app.

"Synchronizing with OS X" is prominently featured/documented in ownCloud's user manual. Hence, a user of ownCloud should expect that this will work. But it won't work for (some?) (ex-)users of Google calendar, if they import their data from Google calendar in ownCloud. Sure you can say, who cares, but if the idea of ownCloud is also about enabling people to set up their own cloud (!) without having a dogfight with RFCs, UIDs, and iCal crash reports, then ownCloud could assign (optionally) new UIDs to all imported VEVENTs of an ICS file in a coherent way, so that third-party clients as iCal don't have trouble with the syncing. Further on, it would be good to make the ICS importer of ownCloud even more robust also to other sloppy violations of the iCalendar standard.

Or ownCloud should only be used by people who are skilled enough to fix such problems.

A user just sees that the import worked, the events are available in ownCloud's Web calendar, but ownCloud fails (!) to sync with iCal. iCal won't notify the user about a problem. Who will be blamed first if a user tries ownCloud the first time? Apple, Google, or ownCloud?

However, the importer has to be more robust and has to check the compliance with the iCalendar standard and otherwise should enforce the compliance!

[tanghus]

then ownCloud could assign (optionally) new UIDs

Not an option:

• If we change the UID, a subsequent import or sync would duplicate the data.
• How should we decide which UIDs are valid and which are not?

We do however assign UIDs if the are missing - at least I do in contacts. I suppose the same goes for calendar @georgehrke ?

but ownCloud fails (!) to sync with iCal.

Other way around ;)

it would be good to make the ICS importer of ownCloud even more robust also to other sloppy violations of the iCalendar standard.

Which would open for more crashes in iCal :smile: Actually there is an option in the newer versions of Sabre\VObject that allows for skipping malformed lines: VObject\Reader::OPTION_IGNORE_INVALID_LINES. I use that in contacts, don't know if it has been implemented in calendar @georgehrke ?

Just out of curiosity I would like to see what UIDs makes iCal crash - then I can send invites to "special friends" :smiley_cat:

[tanghus]

Oh btw: You're more than welcome to contribute with a script to clean up the lousy exports Google creates.

[mhaller]

Google calendar ICS exports include for instance the following different formats of UIDs for VEVENT

UID:283ab21fe78az452fsfa4a314o@google.com

UID:7179652044270346197930806B879E32190366945D81057884564666846D774422475D8
 96566669D058667213D561081981539990587671567

UID:CD55E2D0-5A97-4B76-A54A-93255CEC731F

UID:3D1D06B8-BB7A-4B89-B687-7D030DA636CA-F78401CA-796B-4E7C-BE0B-23331B926D
 7A//hta348b367df36ad6jr5klac3b@group.calendar.google.com

[mhaller]

Ok, you can find the Python code at: https://github.com/mhaller/calfix

Hope, it is useful also for other ownClouders to switch completely to ownCloud!

[tanghus]

Well they're all legal, but excessively long ;)

[joemaller]

I agree with your assessment @tanghus, but it might be a good idea to warn users about the issue @mhaller brings up. I just found out about OwnCloud but we have a lot of Apple Calendar users to support.

[tanghus]

@joemaller @mhaller Maybe a good idea to add it somewhere here https://github.com/owncloud/documentation/blob/master/user_manual/pim/calendar.rst#synchronising-calendars-with-caldav ? We will happily accept pull requests for the documentation.