Issues with bigger contact books

[JamborJan]

I wanted to switch from my current mail providers contact book to radicale on sandstorm. It seems that the whole setup is not stable yet. I have 467 contacts in my address book and experienced this issue. I copied contacts on my client (OS X 10.11.5 stock contacts app) by connecting both - old and new address book and moving them.

• when I copy in batches of e.g. 20 contacts it works like 5 to 10 times, then I get an error message in the OS X log:

20.06.16 20:50:18.544 Contacts[13952]: [CardDAVPlugin-ERROR] -getHomeInfo:[_controller containersAtURLs:{(
    https://sandstorm@api-:443/radicale/owner/addressbook.vcf/
)}] Error Domain=NSURLErrorDomain Code=-1001 "Zeitüberschreitung bei der Anforderung." UserInfo={NSUnderlyingError=0x7fd59c0b6e20 {Error Domain=kCFErrorDomainCFNetwork Code=-1001 "Zeitüberschreitung bei der Anforderung." UserInfo={NSErrorFailingURLStringKey=https://sandstorm@api-/radicale/owner/addressbook.vcf/, NSErrorFailingURLKey=https://sandstorm@api-/radicale/owner/addressbook.vcf/, _kCFStreamErrorCodeKey=-2102, _kCFStreamErrorDomainKey=4, NSLocalizedDescription=Zeitüberschreitung bei der Anforderung.}}, NSErrorFailingURLStringKey=https://sandstorm@api-/radicale/owner/addressbook.vcf/, NSErrorFailingURLKey=https://sandstorm@api-/radicale/owner/addressbook.vcf/, _kCFStreamErrorDomainKey=4, _kCFStreamErrorCodeKey=-2102, NSLocalizedDescription=Zeitüberschreitung bei der Anforderung.}

• when this happens I also cannot open the grain again, the "fortune wheel" which tells that a grain starts keeps turning, nothing in the grain logs, nothing opens. I also cannot sync anymore to the client.
• when I delete everything and start over again and copy everything at once it works at the beginning and I can also open the grain one time, as soon as I change one contact and the sync started, the grain will not properly open anymore. The grain size is still about 5MB but after some waiting time the appears empty
• Once in a while it also shows this small red icon:

In the grains log I find this, not sure if it is connected to the red icon:

sandstorm/sandstorm-http-bridge.c++:480: error: exception = (remote):0: failed: remote exception: remote exception: remote exception: Error: client disconnected
    at ResponseStream.write (server/proxy.js:2321:15)
    at LocalCapWrapper.write (/node_modules/capnp.js:185:28)
stack: 0x44ddb0 0x413850 0x4ce2c0

• downloading a backup doesn't work (takes forever and than the process stops)

All these information are a bit unstructured and unfortunately I cannot provide any dumps and backups as it contains a lot of personal information. Please let me know how I can help tracking down the cause of the issue.

[synchrone]

Ok, so I generated a huge fake .vcf file (thanks to mass whois-data-generator by cadic. big.zip

Will take on the issue next week from a macos.

[synchrone]

As far as I got with my tests, macos' Contacts app can be quite slow to realize it should start uploading CalDAV entries when working with sandstorm-radicale.

As for the performance issue (to the point of seeing this exclamation mark in UI - this has not happen to me on 1k entries address book.

When you have a big file to export - mac starts to bombard the server with PUT requests one-by-one, which can be an issue for the backend.

[ocdtrekkie]

As a note, I just decided to play with this app a little, and imported 935 contacts into it. Shortly thereafter the tab started crashing after a minute or two, causing Edge to ask if I wanted to Recover the page, and with the cursor spinning endlessly.

[synchrone]

Frontend performance is ... well... not quite there. InfCloud codebase is plain horrible, to be honest. The only reason I decided to use it - is because it's the only thing that actually works consistently with CalDAV + CardDAV, directly implementing the protocols from a browser.

[JamborJan]

Thanks for your comments and thoughts on this issue. What is the next step here? Anything I can do or are you @synchrone still working on this?

[synchrone]

Okay, first of all: what @JamborJan reported initially is one backend issue (which I could not reprodce, tbh), and @ocdtrekkie reports a different Frontend issue, which is consistent in bigger address books due to InfCloud being the jQuery mess it is.

Now, how to solve this - is a good question. One could try fiddling around the existing codebase of InfCloud's CardDavMATE in order to introduce pagination, or optimize DOM a little. At the moment I have no time or desire to endeavor in that direction, so I guess we'll leave it, until perhaps another open-source CardDAV web UI implementation would become available.

I'm going to close this issue for now. If the exclamation mark and API access backend issues re-appear (with either Oasis or personal-hosted) - feel free to comment here, and I will look into it again.

[ocdtrekkie]

@synchrone Would it be worth opening another issue on the frontend issue as a reference then?

[synchrone]

Sure, go ahead