Newdle as CalDAV Client

[dilyanpalauzov]

I am very glad to read that CERN switches to open source for its “office work” and contributes to create open source software in the category “Events”.

There are standard protocols suitable for sharing information about events: iCalendar, CalDAV, and possibly JMAP. These protocols are not well supported by big vendors on purpose. In the repositories of newdle, indico and indico-plugins do not mention these terms, apart from newdle/calendar.py and indico/modules/categories/serialize.py:28: """Export the events in a category to iCal. .

Please extend your framework to work with iCalendar and CalDAV. I can provide you with a sample user and password: aaa@aegee.org and abc . It works even with SPNEGO/Kerberos (for aaa@AEGEE.ORG), but also works anonymously (it returns only 401 when username/password do not match). You can edit the personal calendar of the account as you want.

Based on this email address, RFC 6764, and the DNS TXT/SRV records _caldavs._tcp.aegee.org the newdle-server shall be able to determine the bootstrapping URL: https://mail.aegee.org/dav/calendars:444 and then the calendar-home-set. Based on the results, with this query:

curl -XREPORT -H'content-type:application/xml; charset="utf-8"' -HDepth:1 --data '<?xml version="1.0" encoding="utf-8" ?><C:free-busy-query xmlns:C="urn:ietf:params:xml:ns:caldav"><C:time-range start="20210204T140000Z" end="20210305T220000Z"/></C:free-busy-query>' https://mail.aegee.org:444/dav/calendars/user/aaa@aegee.org/ # -u aaa@aegee.org:abc

the server returns the free-busy state.

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CyrusIMAP.org/Cyrus 3.2.5-174-gc44f9d093//EN
BEGIN:VFREEBUSY
DTSTAMP:20210214T125927Z
DTSTART:20210204T140000Z
DTEND:20210305T220000Z
FREEBUSY:20210214T130000Z/20210214T131500Z
END:VFREEBUSY
END:VCALENDAR

As the purpose of Newdle is to find free slots of meetings, please make it a proper CalDAV client, which client by provided any email address as attendee, shows the availability (if the attendee's server permits this, as is the case for aaa@aegee.org) of the participant.

For the record Gnome Evolution can read the free-busy information from the server (either in FBURL, or by EWS) and visualize when all parties have free time. So ideally it shall be possible to allocate a slot for a meeting and book the meeting using either Evolution or Newdle.

[pferreir]

I think integrating with open protocols should be our goal. However, I don't like the idea of storing plaintext user credentials in newdle. I would be in favour of supporting some sort of OAuth-based workflow on a limited (whitelisted) number of providers instead.

[dilyanpalauzov]

The provided example - aaa@aegee.org - works without credentials.

In practice there are different means to insert authentication - GSSAPI, IP-based access and so on: it is up to the server providing free-busy information.

Besides, apart from CALDAV-free-busy-REPORT, there are FBURLS (free-busy urls), described at http://www.calconnect.org/pubdocs/CD0903%20Freebusy%20Read%20URL.pdf . But there is no standard way to detecet the FBURL for a user based only on its email address (but the FBURL can be obtained using LDAP)

[pferreir]

I assumed that most users will not want to have that information provided over an unauthenticated channel. Maybe it's a wrong assumption, though. But in any case, how many users will be in that situation (unless they have their own server)?

And how would you go about using GSSAPI to allow newdle to access your free-busy information? Can you somehow relay a kerberos ticket through the browser? Sorry, I'm clueless on that.

FBURL with a token looks interesting though.

[dilyanpalauzov]

As long as the direction is to make Newdle CalDAV capable, the precise details on Authentication are not highly important from the beginning. Exposing unauthenticated information in a calendar, where the working hours are limited by VAVAILABILITY component, and the rest is just free/busy status, is not soo secret and there is no other universal way to make a system working across organizations.

As a matter of fact, currently users are not entitled to decide, whether they want to offer beyond their organization information about their freebusy status, on any Calendar server, as far as I know.

I think with S4U2Self/S4U2Proxy/proxiable the GSSAPI ticket proxing over the browser shall work, but I never tried it.

There is also an iCalendar proposal for consensus scheduling.

Ideally Newdle shall just be a CalDAV client with web interface, that works for the users within an organization, but somehow tries to work also across organizations.

Trying to achieve everything at the same time (incl. supporting CalDAV Scheduling Inbox, CalDAV Scheduling Outbox, JMAP support, interoperability with EWS, etc) might be too challenging, and might lead to failure. Think on some approach that consists of smaller steps, but leads to a full-featured CalDAV client.

Likewise Indico shall also be converted to a CalDAV client.

CERN has chosen Kopano as Outlook/Exchange alternative and Kopano integrates both CalDAV server with a webclient (generally speaking, I do not know the details of every CalDAV implementation). In theory, every integration of server with client is a vendor lock, where the integration works good, but standard protocols might be sacrificed (because the integration is a priority). This is really not my business, but I would have preferred if CERN has chosen a CalDAV server without integrated web interface,which implements cleanly as much as possible the standard protocols. Then the efforts of CERN shall have gone into providing an open-source CalDAV client, that works with any CalDAV server, because currently there are no good CalDAV clients.

[pferreir]

As long as the direction is to make Newdle CalDAV capable, the precise details on Authentication are not highly important from the beginning.

Well, I believe they are if they will get in the way of users. Or if having to store credentials and usernames is a likely scenario.

As a matter of fact, currently users are not entitled to decide, whether they want to offer beyond their organization information about their freebusy status, on any Calendar server, as far as I know.

Even if that's true, it doesn't seem to respect the "privacy by design" mantra. "It's not so secret" is not something we can guide our actions by in the GDPR era.

There is also an iCalendar proposal for consensus scheduling.

That's very interesting. Thanks for sharing!

Ideally Newdle shall just be a CalDAV client with web interface, that works for the users within an organization, but somehow tries to work also across organizations.

Newdle is a meeting scheduling tool. Having reliable free-busy information is of course super important. But I would never say newdle is a calendaring client at heart. If you want a calendaring client, there are much better applications for that which even run on your desktop (and they can store your credentials if you want them to). What I would like is to get a way to retrieve free-busy information for users who explicitly agree to sharing it, and without the need to store sensitive information on our side (i.e. nothing more than a token). A handful of OAuth-enabled CalDAV providers could be a start, but I don't know of any besides Google Calendar.

Likewise Indico shall also be converted to a CalDAV client.

No, on the contrary. Indico is a source of events. At most it could include a CalDAV server. It's actually something we've already considered and might expand on in the medium to long term.

This is really not my business, but I would have preferred if CERN has chosen a CalDAV server without integrated web interface,which implements cleanly as much as possible the standard protocols.

Stay tuned for news, that's all I can say 😉

[dilyanpalauzov]

A good start would be to make Newdle to skip the Authentication, when the free-busy information server does not require authentication from the CalDAV client (e.g. when the IP is internal). In addition you can make Newdle OAUTH capable, but even then the client shall be able to skip the authentication, when the server (for whatever reason) does not require it.

Let’s say whether the availability/free/busy information can be accessed without authentication can be adjusted using WebDAV ACLs, in particular the CALDAV:schedule-query-freebusy Privilege. I am not aware of any CalDAV client that allows the user to set the privileges on its own calendars. Permitting the users to set for them the CALDAV:schedule-query-freebusy privilege obviously does not violate the privacy by design principle.

What I would like is to get a way to retrieve free-busy information for users who explicitly agree to sharing it, and without the need to store sensitive information on our side (i.e. nothing more than a token). A handful of OAuth-enabled CalDAV providers could be a start, but I don't know of any besides Google Calendar.

I can provide an example, that does not require any authentication. Once Newdle works without authentication, it can be extended to work with authentication. The code doing “boostrapping” from any email address must anyway be written.

Newdle is a meeting scheduling tool.

Sure, like any other comprehensive CalDAV client. What does Newdle offer, which iCalendar/CalDAV - with all their extensions and drafts - do not theoretically provide?

Likewise Indico shall also be converted to a CalDAV client.

No, on the contrary. Indico is a source of events. At most it could include a CalDAV server. It's actually something we've already considered and might expand on in the medium to long term.

Creating a CalDAV server that obeys all possible standards (e.g. the Prefer:minimal header), can scale, provides HA, is utterly hard. As I said, all CalDAV servers, which come with bundled web client, have the problem, that they do not implement the IETF protocols very cleanly.

What does Indico offer, that cannot be mapped to iCalendar/CalDAV?

[pferreir]

A good start would be to make Newdle to skip the Authentication, when the free-busy information server does not require authentication from the CalDAV client (e.g. when the IP is internal).

But that's not very useful to have if only a handful of users will be using it that way, is it? I like starting from prototypes, but even for that there are limits. I don't know many people who own a CalDAV server, let alone one which allows for anonymous access.

I can provide an example, that does not require any authentication. Once Newdle works without authentication, it can be extended to work with authentication. The code doing “boostrapping” from any email address must anyway be written.

Once again, the code doesn't have to be written if there is no hope anyone will ever use it.

Sure, like any other comprehensive CalDAV client. What does Newdle offer, which iCalendar/CalDAV - with all their extensions and drafts - do not theoretically provide?

That's exactly the point: if the set of extensions and drafts you allude to worked in practice, then we wouldn't even need newdle or other similar tools. The fact that there's a market for it kind of proves that it solves a problem which is currently badly addressed. That doesn't make the prospect of implementing a whole pile of calendaring standards very attractive. Forgive me for the pragmatism, but as much as I like the idea of open standards, I don't think they should be implemented just for the sake of it. I would prefer (wild idea) to implement something more lightweight, like OCM, and try to get cloud providers to implement a CardDAV workflow in their platforms, than turning newdle into a glorified CalDAV client which has to deal with a number of login methods, as "standard" as they are.

What does Indico offer, that cannot be mapped to iCalendar/CalDAV?

The way I see it, Indico in an event store. It's not a calendaring application, it's not a meeting scheduling application. If there's an application which you can classify as a source of events, that's Indico. Adding anything which resembles per-user calendar sync would basically turn it into some unmanageable unscalable mess for little gain. I'm totally fine with syncing with specific enterprise calendars (1 provider, e.g. CERN calendar, 1 task syncing every new/updated event), completely against importing personal data from users' calendars. That's simply not the purpose.

[dilyanpalauzov]

I think there are no comprehensive, universal CalDAV clients, because there is no commercial interest in developing such clients. Spending free time just to understand all the iCalendar/CalDAV standards is on its own too much work. Writing (IETF) standards also costs very much time, but is the way to offer universal services. There are tools, that solve one or another calendar problem, which tools are good, but as they do not use standards, the tools are not interoperable. Scheduling a meeting is just one example.

I think I am the only one worldwide who offers CalDAV server with anonymous access: otherwise Apple would have allowed to setup CalDAV account without user name. For such penetrant clients/CUAs the username anonymous@aegee.org works with any password.

As an example in AEGEE there is a tool written from scratch for conference management. Apart from it at https://aegee.org/calendar all events are published as HTML. This is much like https://home.cern/events and https://indico.cern.ch/category/0/calendar .

The AEGEE events can be integrated in any CalDAV client: in DavX5 (Android) just the domain aegee.org as “Special setup” has to be entered and then all events get imported. Likewise in Evolution (New → Collection account, Advanced option: server: mail.aegee.org, no user name), and Thunderbird (with TbSync and Dav-4-TbSync).

As outlined at https://www.oms.aegee.org/wiki/index.php?title=Calendars the events can also be imported as one big iCalendar file, which allows integration e.g. with free on-line calendars (which also offer free mail services).

This allows the following workflow: when non-members subscribe anonymously, they see integrated all the events on their device. When members subscribe, they see in addition internal things and their private calendars, and can choose not to subscribe the public calendars (that are read-only and different tools must be use to change data in public calendars).

Whether Indico

• is converted to a CalDAV server, which is fetched periodically for the latest information, which information is then pushed to a proper CalDAV server, or
• uploads by itself information for events over CalDAV/iCalendar to a CalDAV server

does not make big difference, as long as at the end the information from many sources (private, public/shared) can be presented at the same time in one application. The difference would be only, whether over CalDAV the start/end/description of the event can be changed.

I know CERN is not an Agency for organizing Events and therefore creating software for events is not a priority. Because, as a matter of fact, implementing standards for events in general does not enjoy big interest, there is no good interoperable client-side CalDAV software. Instead good tools exist, that have both some restrictions and cannot be used interchangeably with another tool (that has different restrictions).

[dilyanpalauzov]

Rather than speculating on privacy concerns, why don’t you offer CERN members to Opt-In and expose world-wide, unrestricted, by terms of anonymous access, their free-busy status, stating that if 100 members Opt-In Newdle will be extended to show the freebusy status of those members, when scheduling a meeting?