tonytone commented 12 years ago

Hi there,

I downloaded your plugin for more or less the same reasons why you wrote it; The existing plugin for RoundCube just doesn't perform well at all. Unfortunately I've been having difficulties getting this plugin to run properly. My environment as it is setup at the moment contains an exchange 2007 server for mail a carddav "gateway" and RoundCube loaded with your plugin. This setup appears to work properly with both iCal and Addressbook on OSX Lion connecting to both the exchange calendar and addressbook respectively thru the carddav gateway.

While analyzing the gateway logs the fallowing errors caught my attention;

HTTP/1.1 503 Service Unavailable
Public folders not available: 401 Unauthorized

However it is unclear to me which service the plugin is trying to contact that returns the 503 and if the 401 could still be causing this issue since it is only a warning and the process seems to continue. https://exchange./ews/exchange.asmx returns a login prompt that accepts the same credentials also provided to the plugin and returns an XML as expected. I hope you are able and willing to help me debug this issue I have pasted my logs below.

Thanks in advance! Tony

O/S: FreeBSD 8.2 Mail Server: MS Exchange 2007 Carddav Gateway: DAVMail 3.9.4 Carddav Client: Roundcube-CardDAV-0.2-7 Roundcube: 0.6

davmail error log: [davmail.caldav.CaldavServer] davmail - Connection from /192.168.0.9 on port 1080 [CaldavConnection-17744] davmail - < OPTIONS /users/<user>@<domain>/contacts/ HTTP/1.1 DEBUG [CaldavConnection-17744] davmail - < User-Agent: CardDAV-PHP/0.4.4 DEBUG [CaldavConnection-17744] davmail - < Host: dev.<domain>:1080 DEBUG [CaldavConnection-17744] davmail - < Accept: */* DEBUG [CaldavConnection-17744] davmail - > HTTP/1.1 200 OK DEBUG [CaldavConnection-17744] davmail - > Server: DavMail Gateway 3.9.4-trunk DEBUG [CaldavConnection-17744] davmail - > DAV: 1, calendar-access, calendar-schedule, calendarserver-private-events, addressbook DEBUG [CaldavConnection-17744] davmail - > Date: Mon, 28 Nov 2011 15:24:19 +0000 DEBUG [CaldavConnection-17744] davmail - > Expires: Mon, 28 Nov 2011 15:24:19 +0000 DEBUG [CaldavConnection-17744] davmail - > Cache-Control: private, max-age=0 DEBUG [CaldavConnection-17744] davmail - > Allow: OPTIONS, PROPFIND, HEAD, GET, REPORT, PROPPATCH, PUT, DELETE, POST DEBUG [CaldavConnection-17744] davmail - > Connection: keep-alive DEBUG [CaldavConnection-17744] davmail - > Content-Length: 0 DEBUG [CaldavConnection-17744] davmail - > DEBUG [davmail.caldav.CaldavServer] davmail - Connection from /192.168.0.9 on port 1080 DEBUG [CaldavConnection-19844] davmail - < OPTIONS /users/<user>@<domain>/contacts/ HTTP/1.1 DEBUG [CaldavConnection-19844] davmail - < User-Agent: CardDAV-PHP/0.4.4 DEBUG [CaldavConnection-19844] davmail - < Host: dev.<domain>:1080 DEBUG [CaldavConnection-19844] davmail - < Accept: */* DEBUG [CaldavConnection-19844] davmail - > HTTP/1.1 200 OK DEBUG [CaldavConnection-19844] davmail - > Server: DavMail Gateway 3.9.4-trunk DEBUG [CaldavConnection-19844] davmail - > DAV: 1, calendar-access, calendar-schedule, calendarserver-private-events, addressbook DEBUG [CaldavConnection-19844] davmail - > Date: Mon, 28 Nov 2011 15:24:19 +0000 DEBUG [CaldavConnection-19844] davmail - > Expires: Mon, 28 Nov 2011 15:24:19 +0000 DEBUG [CaldavConnection-19844] davmail - > Cache-Control: private, max-age=0 DEBUG [CaldavConnection-19844] davmail - > Allow: OPTIONS, PROPFIND, HEAD, GET, REPORT, PROPPATCH, PUT, DELETE, POST DEBUG [CaldavConnection-19844] davmail - > Connection: keep-alive DEBUG [CaldavConnection-19844] davmail - > Content-Length: 0 DEBUG [CaldavConnection-19844] davmail - > DEBUG [davmail.caldav.CaldavServer] davmail - Connection from /192.168.0.9 on port 1080 DEBUG [CaldavConnection-21936] davmail - < PROPFIND /users/<user>@<domain>/contacts/ HTTP/1.1 DEBUG [CaldavConnection-21936] davmail - < User-Agent: CardDAV-PHP/0.4.4 DEBUG [CaldavConnection-21936] davmail - < Host: dev.<domain>:1080 DEBUG [CaldavConnection-21936] davmail - < Accept: */* DEBUG [CaldavConnection-21936] davmail - > HTTP/1.1 401 Unauthorized DEBUG [CaldavConnection-21936] davmail - > WWW-Authenticate: Basic realm="DavMail Gateway" DEBUG [CaldavConnection-21936] davmail - > Connection: keep-alive DEBUG [CaldavConnection-21936] davmail - > Content-Length: 0 DEBUG [CaldavConnection-21936] davmail - > DEBUG [CaldavConnection-21936] davmail - < PROPFIND /users/<user>@<domain>/contacts/ HTTP/1.1 DEBUG [CaldavConnection-21936] davmail - < Authorization: ******** DEBUG [CaldavConnection-21936] davmail - < User-Agent: CardDAV-PHP/0.4.4 DEBUG [CaldavConnection-21936] davmail - < Host: dev.<domain>:1080 DEBUG [CaldavConnection-21936] davmail - < Accept: */* DEBUG [CaldavConnection-21936] davmail - Found permanently accepted certificate, hash * DEBUG [CaldavConnection-21936] davmail - executeFollowRedirects(https://exchange.<domain>/owa/) DEBUG [CaldavConnection-21936] davmail - executeFollowRedirects: https://exchange.<domain>/owa/auth/logon.aspx?url=https://exchange.<domain>/owa/&reason=0 redirectCount:1 DEBUG [CaldavConnection-21936] davmail.exchange.ExchangeSession - Form based authentication detected DEBUG [CaldavConnection-21936] davmail.exchange.ExchangeSession - Detected script based logon, redirect to form at https://exchange.<domain>/owa/auth/logon.aspx?replaceCurrent=1&url=https%3a%2f%2fexchange.<domain>%2fowa%2f DEBUG [CaldavConnection-21936] davmail - executeFollowRedirects(https://exchange.<domain>/owa/auth/logon.aspx?replaceCurrent=1&url=https%3a%2f%2fexchange.<domain>%2fowa%2f) DEBUG [CaldavConnection-21936] davmail - executeFollowRedirects(https://exchange.<domain>/owa/auth/owaauth.dll) DEBUG [CaldavConnection-21936] davmail - executeFollowRedirects: https://exchange.<domain>/owa/ redirectCount:1 WARN [CaldavConnection-21936] davmail.exchange.ExchangeSession - Public folders not available: 401 Unauthorized at https://exchange.<domain>/public/ DEBUG [CaldavConnection-21936] davmail.exchange.ExchangeSession - Current user email is <user>@<domain>, alias is <user>, mailPath is /exchange/<user>@<domain>/ on Exchange2007 DEBUG [CaldavConnection-21936] davmail.exchange.ExchangeSession - Inbox URL: https://exchange.<domain>/exchange/<user>@<domain>/Inbox Trash URL: https://exchange.<domain>/exchange/<user>@<domain>/Deleted Items Sent URL: https://exchange.<domain>/exchange/<user>@<domain>/Sent Items Send URL: https://exchange.<domain>/exchange/<user>@<domain>/##DavMailSubmissionURI##/ Drafts URL: https://exchange.<domain>/exchange/<user>@<domain>/Drafts Calendar URL: https://exchange.<domain>/exchange/<user>@<domain>/Calendar Tasks URL: https://exchange.<domain>/exchange/<user>@<domain>/Tasks Contacts URL: https://exchange.<domain>/exchange/<user>@<domain>/Contacts Outbox URL: https://exchange.<domain>/exchange/<user>@<domain>/Outbox Public folder URL: /public/ DEBUG [CaldavConnection-21936] davmail.exchange.ExchangeSession - Session davmail.exchange.dav.DavExchangeSession@120bf2c created DEBUG [CaldavConnection-21936] davmail.exchange.ExchangeSession - Created new session: davmail.exchange.dav.DavExchangeSession@120bf2c ERROR [CaldavConnection-21936] davmail - java.lang.NullPointerException java.lang.NullPointerException at java.io.StringReader.<init>(StringReader.java:33) at davmail.exchange.XMLStreamUtil.createXMLStreamReader(XMLStreamUtil.java:155) at davmail.caldav.CaldavConnection$CaldavRequest.parseXmlBody(CaldavConnection.java:1458) at davmail.caldav.CaldavConnection$CaldavRequest.<init>(CaldavConnection.java:1293) at davmail.caldav.CaldavConnection.handleRequest(CaldavConnection.java:208) at davmail.caldav.CaldavConnection.run(CaldavConnection.java:170) DEBUG [CaldavConnection-21936] davmail - > HTTP/1.1 503 Service Unavailable DEBUG [CaldavConnection-21936] davmail - > Server: DavMail Gateway 3.9.4-trunk DEBUG [CaldavConnection-21936] davmail - > DAV: 1, calendar-access, calendar-schedule, calendarserver-private-events, addressbook DEBUG [CaldavConnection-21936] davmail - > Date: Mon, 28 Nov 2011 15:24:23 +0000 DEBUG [CaldavConnection-21936] davmail - > Expires: Mon, 28 Nov 2011 15:24:23 +0000 DEBUG [CaldavConnection-21936] davmail - > Cache-Control: private, max-age=0 DEBUG [CaldavConnection-21936] davmail - > Content-Type: text/plain;charset=UTF-8 DEBUG [CaldavConnection-21936] davmail - > Connection: close DEBUG [CaldavConnection-21936] davmail - > Content-Length: 30 DEBUG [CaldavConnection-21936] davmail - > DEBUG [CaldavConnection-21936] davmail.caldav.CaldavConnection - > java.lang.NullPointerException

roundcube error log:

`[28-Nov-2011 16:16:21] PHP Fatal error: Uncaught exception 'Exception' with message 'String could not be parsed as XML' in /usr/local/www/roundcube/plugins/carddav/carddav_addressbook.php:408 Stack trace:

0 /usr/local/www/roundcube/plugins/carddav/carddav_addressbook.php(408): SimpleXMLElement->__construct('')

1 /usr/local/www/roundcube/plugins/carddav/carddav.php(240): carddav_addressbook->carddav_addressbook_sync()

2 /usr/local/www/roundcube/plugins/carddav/carddav.php(414): carddav->carddav_addressbook_sync(65, false)

3 [internal function]: carddav->carddav_server_save()

4 /usr/local/www/roundcube/program/include/rcube_plugin_api.php(296): call_user_func(Array)

5 /usr/local/www/roundcube/index.php(239): rcube_plugin_api->exec_action('plugin.carddav-...')

6 {main}

thrown in /usr/local/www/roundcube/plugins/carddav/carddav_addressbook.php on line 408`

christian-putzke commented 12 years ago

I'll have a closer look at the issue this evening

tonytone commented 12 years ago

Thanks a lot in the meantime I've updated the davmail gateway to the latest version 3.9.6 build 1843, however issue remains.

christian-putzke commented 12 years ago

I'm not sure why davmail throws a null pointer exception but it seems that davmail has a different handling for the CardDAV-XML-Requests. I've never tested my CardDAV-PHP-Client with davmail but for memotoo, apple addressbook server and sabreDAV I had to make a view changes. Maybe you can provide me a test account on your server so that I can try to figure out the error directly.

tonytone commented 12 years ago

Well theres a big difference in the propfind request send by CardDAV-PHP as opposed to AddressBook. The nullpointer in davmail suggests davmail expects an xml but does not receive any. This would be correct since the carddav-php request doesnt have any body. Is this intended carddav-php behavior or could something be breaking resulting in a partial request? I tried to figure out where and how this request is build up in cardav-php but wasn't able to.

Unfortunately I am unable to provide access to our network although that would have probably made this a lot easier. But hopefully my findings can point you in the right direction?

< PROPFIND /users/user@domain/contacts/ HTTP/1.1 < User-Agent: CardDAV-PHP/0.4.4 < Host: dev.domain:1080 < Accept: */*

< PROPFIND /principals/users/user%40domain/ HTTP/1.1 < Host: dev.domain:1080 < User-Agent: AddressBook/6.1 (1062) CardDAVPlugin/196 CFNetwork/520.2.5 Mac_OS_X/10.7.2 (11C74) < Content-Length: 439 < Accept: */* < Accept-Language: en-us < Accept-Encoding: gzip, deflate < Content-Type: text/xml < Depth: 0 < Connection: keep-alive < `<?xml version="1.0" encoding="UTF-8"?>

<A:principal-collection-set/>
<A:principal-URL/>
<A:resource-id/>
<A:supported-report-set/>

</A:prop> </A:propfind>`

tonytone commented 12 years ago

ok i think i've figured it out somewhat but im still having problems getting that solution implemented as I am not familiar with this level op coding. The fallowing xml should be send alongside the first propfind to get the servers properties; `

</A:prop> </A:propfind>`

Then (from what i understand from the code) after the server was added to the database the first sync should be performed by means of a second propfind request containing the fallowing xml and search header; `<?xml version="1.0" encoding="UTF-8"?>

<A:current-user-privilege-set/>
<A:displayname/>
<B:max-image-size xmlns:B="urn:ietf:params:xml:ns:carddav"/>
<B:max-resource-size xmlns:B="urn:ietf:params:xml:ns:carddav"/>
<C:me-card xmlns:C="http://calendarserver.org/ns/"/>
<A:owner/>
<C:push-transports xmlns:C="http://calendarserver.org/ns/"/>
<C:pushkey xmlns:C="http://calendarserver.org/ns/"/>
<A:quota-available-bytes/>
<A:quota-used-bytes/>
<A:resource-id/>
<A:resourcetype/>
<A:supported-report-set/>
<A:sync-token/>

</A:prop> </A:propfind>`

Search: SELECT "http://schemas.microsoft.com/exchange/permanenturl","DAV:getetag","urn:schemas:httpmail:subject","http://schemas.microsoft.com/mapi/proptag/x10f3001f","urn:schemas:calendar:instancetype","DAV:displayname" FROM SCOPE('Shallow TRAVERSAL OF "/exchange/user@domain/Contacts"') WHERE ("http://schemas.microsoft.com/exchange/outlookmessageclass" = 'IPM.Contact' And "DAV:isfolder" = false And "DAV:ishidden" = false And "http://schemas.microsoft.com/exchange/outlookmessageclass" = 'IPM.Contact')

I've tried hacking in the first xml which proved my ideas about the initial nullpointer and this resulted in exactly the same response as was given to AddressBook; `<?xml version="1.0" encoding="UTF-8"?>

/principals/users/@

/users/@/contacts/

@

HTTP/1.1 200 OK` However i hard-coded it in the get() inside the carddav_backend.php and this is offcourse not the correct spot for it in your source. Would you be willing to help me implement these changes?

christian-putzke commented 12 years ago

I'll help you as much as I can to implement the changes in my plugin. But at the moment I haven't much time. I'm sorry. Let me know if you get any further results.

tonytone commented 12 years ago

Well ive implemented some changes so I am receiving the contacts from DAVMail now. However now that the contacts are in the database I noticed a few other problems;

Autocomplete only returns the label of the carddav addressbook and not the contacts themselves. It seems that the get_search_set() function generates the search filter for the query like A) in stead of a proper AND bla like bla OR bla like bla.

[08-Dec-2011 17:57:13] MDB2 Error: syntax error (-2): _doQuery: [Error message: Could not execute statement] [Last executed query: SELECT * FROM carddav_contacts WHERE user_id = '1' A) ORDER BY name ASC ] [Native code: 1064] [Native message: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'A) ORDER BY name ASC' at line 7]

Search does not return any carddav results at all Same as above
Syncing does not seem to work either Havent looked into this yet
Changing the addressbooks label results in an error Havent looked into this yet
If contacts have no firstname lastname or email set then no label is displayed in the contactlist (in this case I would like to display the Company name) More of a wish then a bug I guess

I also had to make minor changes to rcube_vcard.php to make cellnumber and aniversary show op correctly, nickname however, doesnt seem to be present in the the database vcard even though it does have a value.

Al the functionalities named above that are applicable to an LDAP addressbook as well, work fine for LDAP addressbooks, except maybe the last one which could be in addressbook's logic rather than your plugin. I understand that you are busy, so ill keep fiddling around untill i get at least these basic functionalities to work properly. But if you have some pointers as to where in your source i could find the code that relates to those functionalities that would be a big help, because to be honest, even though theres some progress, im in WAY over my head here hehe.

christian-putzke commented 12 years ago

Thanks for the huge report!

For me the autocomplete works fine with all contacts from two different CardDAV-Sources. Which version are you using?
Same as 1. works fine for me
Can't say anything about this. But if the first initial import for the CardDAV-Contacts work, the sync should work as well
The adressbooks label can't be changed. I've never implemented such a function ;)
The label for contacts displayed in the addressbook depends on roundcoube. I'll just use the internal hooks to display the CardDAV-Contacts so you should report that issue on roundcube.net
If you change something in the original roundcube source like the rcube_vcard.php you should send your changes also to the roundcube developers itself

Best regards, Christian

christian-putzke commented 12 years ago

No response since 2 months -> close

christian-putzke / Roundcube-CardDAV

nullpointer exception when saving configuration #3

0 /usr/local/www/roundcube/plugins/carddav/carddav_addressbook.php(408): SimpleXMLElement->__construct('')

1 /usr/local/www/roundcube/plugins/carddav/carddav.php(240): carddav_addressbook->carddav_addressbook_sync()

2 /usr/local/www/roundcube/plugins/carddav/carddav.php(414): carddav->carddav_addressbook_sync(65, false)

3 [internal function]: carddav->carddav_server_save()

4 /usr/local/www/roundcube/program/include/rcube_plugin_api.php(296): call_user_func(Array)

5 /usr/local/www/roundcube/index.php(239): rcube_plugin_api->exec_action('plugin.carddav-...')

6 {main}