Can't sync CardDAV with OSX/iOS

[jlongman]

Once the Cloud Sync for CardDAV button is forced displayed (see #7) in version 1.1.1 and the properties are entered in to OSX and iOS contacts configurations the account is not able to be authenticated. (By contrast the CalDAV calendars DO work correctly from the Cloud Sync-based configuration).

I'm running iOS 9 and OS X 10.11.3, radicale 1.1.1 with latest (i.e. the http auth updated) sandstorm.

• OS X responds with "unable to verify account name or password" - and possibly accountsd crashes in the background, although that's clearly beyond the scope of this.
• iOS claims that it "Cannot Connect using SSL", and offers to set it up without SSL. If you click continue it says "CardDAV account verification failed".

Note: If I hit the URL provided then the VCF cards entered into radicale through the web interface are downloaded and imported into the local address book on OSX, but not sync'able.

It appears that the radicale-side configuration of this with OSX/iOS is complex, related radicale issues:

• https://github.com/Kozea/Radicale/issues/256
• https://github.com/Kozea/Radicale/issues/269

I still have to go through the above - but i get the impression some people have gotten it to work at some point. I suspect there's some special munging/specific configuration required for authorization and paths with Apple's CardDAV flavour.

I do notice for example that the path in the URL is '/radicale/owner/addressbook.vcf' - and in the last version the username was 'owner' and not 'radicale'. Changing this to 'sandstorm' did not rectify the problem.

Note: thanks for this, I've been a fan of radicale for a while, and sandstorm and this hits a sweet spot for the way I want to deploy it. Great work!

[synchrone]

Thank you for the bug report.

I was able to reproduce it on a local dev server (without SSL, I believe this might be a separate issue, so let's keep things simple for now).

I don't get why iOS reports a failure. I'm trying to paste the URL provided directly into the Server input field in iOS interface. iOS does 1 (one) request to PROPFIND at the URL specified, and reports failure. Would be super helpful to get it's debug log somehow.

scratch.txt is an nginx debug-log for that. Nothing special there, not even a prior /.well-known request.

I will check the issues you linked, thank you for that.

URL path has nothing to do with it, since Radicale is set up at /radicale prefix, and that hasn't changed since v1. So, owner is the username. I don't think that has to do with the issue.

[synchrone]

I have found the following:

RFC4791, Section 5.1 requires a Calendar endpoint to respond with at least calendar-access capabilities in DAV: header.

In the abovementioned nginx log, we can see Radicale dutifully returning those from uwsgi:

2016/02/24 12:41:28 [debug] 14#0: *2 http uwsgi header: "DAV: 1, 2, 3, calendar-access, addressbook, extended-mkcol"

Sandstorm's proxy.js (here we meet again, Mr. NodeJS-Mangle-Around-My-HTTP) constructs DAV: header on it's own.

I quickly got lost in meteor.js promises, and could not verify that if we "fix" that into complying with the RFC iOS starts working. Hopefully @paulproteus can help us here.

[paulproteus]

Oh, oops. cc: @kentonv - it seems we were mistaken in believing that CalDAV requires no HTTP header changes. Specifically, we need to support some new entries in the DAV: header.

[paulproteus]

If @mnutt feels like taking a look at implementing this, that'd be super hugely awesome. If not, then I can see how soon someone in Sandstorm Core Dev-land can get to it.

[mnutt]

I think the question is if we think we can enumerate all of the DAV capabilities now or if it makes more sense to let the DAV header be essentially an array of strings. Adding a new capability or two is pretty trivial, though.

[paulproteus]

Along those lines, if we continue to add capabilities one by one, it seems smart for sandstorm-http-bridge to log a message if it can't convert the DAV header it receives into a Cap'n Proto message. That would have increased the speed by which this issue was discovered.

[synchrone]

More importantly, it seems that iOS does that OPTIONS request without credentials... which is pretty much a deal breaker, since token is used for routing.

I patched server.js to log request headers around those lines

{ host: 'api-73bbf4d41660c3488ce5241c6dcd46a0.mymachine:6080',
  'accept-encoding': 'gzip, deflate',
  accept: '*/*',
  'content-length': '0',
  connection: 'keep-alive',
  'accept-language': 'ru',
  'user-agent': 'iOS/9.2.1 (13D15) accountsd/1.0' }

So yeah, this does not completely prevent the feature from working, but damages the UX for iOS users even more.

[mnutt]

Does iOS actually care about the response? I have seen other servers that do OPTIONS unauthenticated, get back the standard Sandstorm CORS response, and then merrily go about doing authenticated PROPFINDs.

[synchrone]

Yes. I just patched https://github.com/sandstorm-io/sandstorm/blob/master/shell/server/proxy.js#L960 into adding a hardcoded DAV with CalDAV capability and the iOS error went away.

[mnutt]

It may be interesting to test if a maximalist approach (sandstorm automatically says it supports everything) worked. Unfortunately I imagine it breaks some clients to tell them we support something we don't?

[synchrone]

Okay, with sandstorm 0.149 released and #11 merged, I can confirm the issue is solved for local dev environment.

That said, HTTPs scenario I have not tested yet.

[kentonv]

The update is approved in the app market, but you'll need to un-embargo it for it to go live. :)

[jlongman]

So using 1.1.1+v8? (todays version, just had another update) and a latest sandstorm, I can't connect using OSX, user authentication problems, but iOS I can (working around issues in #13, just created).

Note that upgrading the version or restarting the app in sandstorm appears to change the host (e.g. api-FOOebe98562daafe177884116ec19fd4.oasis.sandstorm.io changes to api-BARe1716ec19fd478841ebe98562daaf.oasis.sandstorm.io) as well as the password changes to a random value. This is also true if the app and/or sandstorm is updated (obviously, since it restarts).

This breaks mobile device access rather easily since the host and login credentials are no longer valid.

[mnutt]

@jlongman as long as the randomly-generated host is used in the first (5?) minutes after it is generated, it will continue to be valid. Most likely something is causing the credentials to not be sent in the first place and so the host expires.

[kentonv]

@jlongman To be clear, it is expected that you see new credentials every time, but that doesn't mean the old credentials don't work anymore. If the old credentials worked once they will continue to work until explicitly revoked. The password you see actually isn't stored anywhere on the server (only a hash is stored), therefore it would be impossible to display the same credentials every time, hence new ones are generated.

[synchrone]

I'm afraid I cannot work on OSX issues, since I don't have a such computer available.

Would be cool if someone could use this package with nginx debug log enabled (I can provide an SPK) and attached the .pcap file together with the logs, so that I could figure out what went wrong during the DAV negotiation.

[synchrone]

@korchasa was kind enough to agree to try it on OSX in debug mode with v9-debug.spk.zip

[synchrone]

@jlongman: are you using Manual or Advanced connection modes? We were able to set up syncing using Advanced mode.

[synchrone]

In Manual mode, OSX 10.11 Calendar does service-discovery.

1. It treats Server field only as domain name, extracting it from a URL if it was given. Actual path on the URL does not matter, as it seems.
2. The port is not taken from the URL, so it tries sequentially: 433, 8433, 80, 8080 and 8008.

3. Each port is tried with those PATH components using a PROPFIND method.

   • /.well-known/caldav
   • /
   • /caldav/v2
   • /principals/users/_username_
   • /principals/
   • /dav/principals

   Upon receiving 401 Unauthorized there, it will retry the request with HTTP Basic using credentials given by the user.

4. Unauthenticated OPTIONS to CalDAV url is done much like iOS, but we already support that.

So, to answer @jlongman: I will assume it does not work on OSX because you are using vagrant-spk, which binds to :6080, which OSX will simply not even try to connect to. If you want to use Radicale hosted in Sandstorm on a custom port - consider setting up via Advanced

[jlongman]

(Sorry, was without my laptop for a couple of days.)

I'm kind of confused because of people use sandstorm in different ways.

• To be clear the radicale instance is on oasis.sandstorm.io, so I'm not sure how I can change the port its run under, if that's what you're suggesting, rather than changing the client configuration,
• In the Contacts.app CardDAV configuration UI, set to CardDAV and Advanced, I have entered all the information that was in the URL (there is no place to enter an URL) and changed the port configuration variously between 443, etc, and I still have an authentication problem.

I don't know that this changes the resolution, maybe it's just not possible to fix it because the OSX Contacts.app application is stupid.

[Sorgrum]

Hey, not sure if anyone is still having this problem, but I was able to get Radicale on my Contacts.app on OS X.

I went to System Preferences

-> Internet Accounts
-> Add Other Account...
-> CardDav account
    -> Account Type: Advanced
        -> User Name: yourusername
        -> Password: yourpassword
        -> Server Address: http://contacts.domain.com
        -> Server Path: /caldav.php/username/
        -> Port: 80
        -> Uncheck Use SSL

Again, new to this, so sorry if this isn't relevant.

Update: On further inspection. When trying to actually modify/add/remove and contacts, the entire Contacts app crashes

[synchrone]

@Sorgrum thank you for a workaround tip. Advanced mode is somewhat more tedious work for the users. I'd like to have Manual mode fixed, which is currently tracked as #23.

[rajid]

Even configuring macos High Sierra contacts as described above, doesn't work. It appears to work, but if you also sync with something else, like an iphone, you'll see that the two are not actually seeing the same entries. Creating a contact on the macos side doesn't even send the information to the server at all. It must be storing it somewhere locally on the machine.

In short, contacts/addressbook in radicale still doesn't work from MacOS X, High Sierra.

[rajid]

Ok, after ignoring the problem for about 30 minutes, they are all (mac, iphone, ipad) now seeing the same address book entries and the bogus ones on the mac have been replaced by the correct ones. Very strange!