pimutils / vdirsyncer

📇 Synchronize calendars and contacts.
https://vdirsyncer.pimutils.org/
Other
1.57k stars 162 forks source link

vdirsyncer finds no contacts in Zimbra address book #83

Closed davidfraser closed 10 years ago

davidfraser commented 10 years ago

I can successfully synchronize contacts with Zimbra using CardDAV with various other programs (CardDAVSync for Android, cadaver, webdav-sync), but vdirsyncer doesn't find any contacts.

Config (actual email, server etc modified):

[general]
status_path = ~/.vdirsyncer/status/
processes = 1

[pair work_contacts]
a = work_contacts_local
b = work_contacts_remote

conflict_resolution = b wins

[storage work_contacts_local]
type = filesystem
path = ~/work/contacts/Contacts/
fileext = .vcf
create = True
encoding = utf-8

[storage work_contacts_remote]
type = carddav
url = https://zimbra.hostname.com/dav/user@server.com/Contacts/
auth = basic
username = user@server.com

Logs (likewise redacted):

debug: Using 1 processes.
debug: Not using multiprocessing.
Server password for user@server.com at the resource https://zimbra.hostname.com/dav/user@server.com/Contacts/: Syncing work_contacts
debug: OPTIONS https://zimbra.hostname.com/dav/user@server.com/Contacts/
debug: {'Depth': 1, 'Content-Type': 'application/xml; charset=UTF-8', 'User-Agent': 'vdirsyncer'}
debug: None
debug: Sending request...
debug: 200
debug: {'date': 'Thu, 03 Jul 2014 13:25:28 GMT', 'dav': '1, 2, 3, access-control, calendar-access, calendar-schedule, version-control, addressbook, extended-mkcol, calendar-proxy, calendarserver-principal-property-search', 'content-length': '0', 'allow': 'MKCOL, REPORT, MOVE, COPY, HEAD, POST, GET, DELETE, ACL, MKCALENDAR, PROPFIND, LOCK, OPTIONS, UNLOCK, PUT, PROPPATCH'}
debug: 
debug: REPORT https://zimbra.hostname.com/dav/user@server.com/Contacts/
debug: {'Depth': 'infinity', 'Content-Type': 'application/xml; charset=UTF-8', 'User-Agent': 'vdirsyncer'}
debug: <?xml version="1.0" encoding="utf-8" ?>
            <C:addressbook-query xmlns:D="DAV:"
                    xmlns:C="urn:ietf:params:xml:ns:carddav">
                <D:prop>
                    <D:getetag/>
                </D:prop>
                <C:filter>
                    <C:comp-filter name="VCARD"/>
                </C:filter>
            </C:addressbook-query>
debug: Sending request...
debug: 207
debug: {'date': 'Thu, 03 Jul 2014 13:25:28 GMT', 'transfer-encoding': 'chunked', 'dav': '1, 2, 3, calendar-access, calendar-proxy, calendarserver-principal-property-search, access-control, addressbook', 'content-type': 'text/xml;charset=UTF-8'}
debug: <?xml version="1.0" encoding="UTF-8"?>

<D:multistatus xmlns:D="DAV:"/>
untitaker commented 10 years ago

Is this URL directly pointing to an addressbook, or do you have multiple ones?

davidfraser commented 10 years ago

Directly to an address book, but I've also tried specifying a collection with one item, and multiple items, and using from b for the collection

untitaker commented 10 years ago

Not sure what is happening here. Can you try https://github.com/untitaker/vdirsyncer/pull/84 and see if it fixes your problem?

pip2 install --user --force-reinstall git+https://github.com/untitaker/vdirsyncer.git#zimbra
davidfraser commented 10 years ago

Nope, sorry :( 500 error for that request:

debug: Sending request...
debug: 200
debug: {'date': 'Thu, 03 Jul 2014 13:46:13 GMT', 'dav': '1, 2, 3, access-control, calendar-access, calendar-schedule, version-control, addressbook, extended-mkcol, calendar-proxy, calendarserver-principal-property-search', 'content-length': '0', 'allow': 'MKCOL, REPORT, MOVE, COPY, HEAD, POST, GET, DELETE, ACL, MKCALENDAR, PROPFIND, LOCK, OPTIONS, UNLOCK, PUT, PROPPATCH'}
debug: 
debug: REPORT https://zimbra.sjsoft.com/dav/davidf@j5int.com/Contacts/
debug: {'Depth': 'infinity', 'Content-Type': 'application/xml; charset=UTF-8', 'User-Agent': 'vdirsyncer'}
debug: <?xml version="1.0" encoding="utf-8" ?>
            <C:addressbook-query xmlns:D="DAV:"
                    xmlns:C="urn:ietf:params:xml:ns:carddav">
                <D:prop>
                    <D:getetag/>
                </D:prop>
                <C:filter>
                </C:filter>
            </C:addressbook-query>
debug: Sending request...
debug: 500
debug: {'date': 'Thu, 03 Jul 2014 13:46:13 GMT', 'content-length': '0'}
debug: 

traceback:

Traceback (most recent call last):
  File "vdirsyncer/vdirsyncer/cli.py", line 333, in sync_collection
    force_delete=status_name in force_delete
  File "vdirsyncer/vdirsyncer/sync.py", line 129, in sync
    list_b = prepare_list(storage_b, b_href_to_status)
  File "vdirsyncer/vdirsyncer/sync.py", line 66, in prepare_list
    for href, etag in storage.list():
  File "vdirsyncer/vdirsyncer/storage/dav.py", line 417, in _list
    headers=headers
  File "vdirsyncer/vdirsyncer/storage/dav.py", line 203, in request
    session=self._session, **self._settings)
  File "vdirsyncer/vdirsyncer/utils/__init__.py", line 213, in request
    r.raise_for_status()
  File "requests/models.py", line 795, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
HTTPError: 500 Server Error: Server Error
untitaker commented 10 years ago

Maybe completely removing the filter tag works then? I think i need to get my hands on a copy of Zimbra to integrate into the testing environment.

davidfraser commented 10 years ago

Zimbra is very heavy ... I don't think you want to install it just for testing this.

It seems very strange that Zimbra is returning this empty set, but on the other hand the other clients work with it... I did some debugging and comparison with pyCardDav (which works) - they do a general PROPFIND with Accept: / and Depth: 1 and that seems to return items if I put it into vdirsyncer, rather than doing an addressbook-query. But that's strange, and I can't see what's missing from our addressbook-query and Zimbra's supposed to support it. :(

untitaker commented 10 years ago

@dmfs Is the behavior with PROPFIND described above implemented because of Zimbra?

davidfraser commented 10 years ago

http://www.ietf.org/mail-archive/web/vcarddav/current/msg01649.html seems to indicate that Zimbra may behave like this (but no replies)

dmfs commented 10 years ago

CardDAV-Sync for Android doesn't use addressbook-query either. I didn't see any benefits in using it. We either use a regular PROPFIND or a sync-collection REPORT if supported. In contrast to calendar-query it usually doesn't make any sense to filter the elements in an address book. As of RFC 6352 an address book collection supports only address resources. Of course that includes other formats than VCard, but I've not seen any server that actually supports other formats. We filter the elements by content-type after we received the list.

untitaker commented 10 years ago

@davidfraser I saw your fork contained a commit re duplicate hrefs. Is it still relevant?

untitaker commented 10 years ago

Can you try again with the latest commit on the zimbra branch, @davidfraser?

davidfraser commented 10 years ago

The latest commit works, but I had to set Depth to 1 otherwise I got a 403 Forbidden with a prop-finite-depth error. Thanks!

The commit re duplicate hrefs was for Zimbra calendar - haven't investigated more but got lots of these (I wonder if they're for recurring appointments?)

untitaker commented 10 years ago

Unfortunately it seems like we can't merge this anytime soon, since it seems to break Radicale.

untitaker commented 10 years ago

@dmfs, i think CardDAV sync for Android then also breaks for Radicale-git with database backend.

dmfs commented 10 years ago

We don't have any reports about problems so far. What's the problem? Does it return an error? How does the request you send look like? How does the response look like?

untitaker commented 10 years ago

See https://travis-ci.org/untitaker/vdirsyncer/jobs/29148128. Radicale simply crashes for reasons i am not even trying to comprehend.

dmfs commented 10 years ago

What was the content of the last request? If I read the log correctly it was empty:

debug: PROPFIND http://127.0.0.1/bob/test.vcf/
debug: {'Depth': 1, 'Content-Type': 'application/xml; charset=UTF-8', 'User-Agent': 'vdirsyncer'}
debug: None

I think that's what None indicates here.

untitaker commented 10 years ago

Yes, precisely. As far as i can see that's what pycarddav does. I'll reread the RFC.

dmfs commented 10 years ago

What's the purpose of an empty PROPFIND? Some servers probably just return a default multistatus response (like an index) but that's not specified and an empty PROPFIND is not valid, afaik.

geier commented 10 years ago

I'll have to admit that I didn't read the RFC properly when implementing this in pycarddav. I poked around a bit and stopped when it seemed to work with all CardDAV servers I had access to.

geier commented 10 years ago

BTW the RFC says:

A client may choose not to submit a request body. An empty PROPFIND request body must be treated as a request for the names and values of all properties.

I'm on the phone, so cannot properly look into this atm.

untitaker commented 10 years ago

Just pushed a fix to the zimbra branch, now using a proper request body for the PROPFIND. @davidfraser Can you try this one too, just to make sure?

davidfraser commented 10 years ago

@unittaker that fixes the synchronization, thanks! I have a problem remaining with address books whose name has a space in them - but will file a separate ticket for that in due course

untitaker commented 10 years ago

Done. Please file a new issue for the addressbook-issue and eventually also re duplicate hrefs.

untitaker commented 10 years ago

Just released 0.2.1.