Open Zeriuno opened 9 years ago
Thanks for checking back in with results and opening a detailed issue :) I promise to check this out this weekend!
I have to thank you, rhodey, (and a few people at my hosting provider too, actually) for helping troubleshooting these issues with Zimbra. Take all the time you need!
@Zeriuno looking at these logs now it seems that you're making it even less far in the server tests than before :[ as in the logs from #73. Are you using the same WebDAV host URL for these tests as you were before for #73?
In #73 the situation was:
The situation in these two new logs seems to be:
OR
If you're sure that you're using the same URL for the tests as before then it would seem that the Zimbra server you're trying to work with has since changed its configuration. Here is some curl stuff you can use to help the testing along...
$ curl -k -i -u <username>:<password> -X PROPFIND https://<WebDAV host>/.well-known/carddav --upload-file - -H "Depth: 0" <<end
<?xml version="1.0"?>
<a:propfind xmlns:a="DAV:">
<a:prop><a:current-user-principal/></a:prop>
</a:propfind>
end
The above curl request mimics the first test run by the Flock client which is a test for the current-user-principal
property. The trailing /.well-known/carddav
is something called a well-known URI and is defined in rfc 6764. Well-known URIs are nice because they make it possible for a WebDAV client to work with a WebDAV server without having to ask the user for a bunch of configuration information.
The correct response to a PROPFIND
request on a well-known URI is a redirect to the actual resource. For example on OwnCloud /.well-known/carddav
maps to something like .../remote.php/carddav' and
/.well-known/caldavmaps to something like
.../remote.php/caldav'. Check out the example request and response to one of my testing Darwin Calendar Server instances below...
$ curl -k -i -u coffee00@flock:test -X PROPFIND http://55.55.55.55/.well-known/carddav --upload-file - -H "Depth: 0" <<end
<?xml version="1.0"?>
<a:propfind xmlns:a="DAV:">
<a:prop><a:current-user-principal/></a:prop>
</a:propfind>
end
HTTP/1.1 301 Moved Permanently
Date: Mon, 01 Dec 2014 21:15:02 GMT
Content-Length: 165
Location: http://55.55.55.55/
Content-Type: text/html;charset=utf-8
Server: Twisted/13.2.0 TwistedWeb/9.0.0+r11995
<html><head><title>Moved Permanently</title></head><body><h1>Moved Permanently</h1><p>Document moved to http://55.55.55.55/.</p></body></html>
Darwin Calendar Server really just does WebDAV stuff so there is no special endpoint for CardDAV vs CalDAV requests, this response tells us that for everything CardDAV we can just talk to the root endpoint at http://55.55.55.55/
. So now that we know where to send our CardDAV requests we try the current-user-principal
PROPFIND request again but this time with the new target set to http://55.55.55.55/
$ curl -k -i -u coffee00@flock:test -X PROPFIND http://55.55.55.55/ --upload-file - -H "Depth: 0" <<end
<?xml version="1.0"?>
<a:propfind xmlns:a="DAV:">
<a:prop><a:current-user-principal/></a:prop>
</a:propfind>
end
HTTP/1.1 207 Multi-Status
Date: Mon, 01 Dec 2014 21:15:33 GMT
ETag: "44925-1000-546E5445"
Accept-Ranges: bytes
Content-Type: text/xml
Last-Modified: Thu, 20 Nov 2014 20:51:17 GMT
Content-Length: 364
Server: Twisted/13.2.0 TwistedWeb/9.0.0+r11995
DAV: 1, access-control, calendar-access, calendar-schedule, calendar-auto-schedule, calendar-availability, inbox-availability, calendarserver-private-events, calendarserver-private-comments, calendar-query-extended, calendar-default-alarms, calendarserver-partstat-changes, calendar-no-timezone, addressbook, extended-mkcol, calendarserver-principal-property-search, calendarserver-principal-search, calendarserver-home-sync
Vary: accept-encoding
<?xml version='1.0' encoding='UTF-8'?>
<multistatus xmlns='DAV:'>
<response>
<href>/</href>
<propstat>
<prop>
<current-user-principal>
<href>/principals/__uids__/COFFEE00%40FLOCK/</href>
</current-user-principal>
</prop>
<status>HTTP/1.1 200 OK</status>
</propstat>
</response>
</multistatus
And there ya go, the current-user-principal
property we asked for was returned and as expected it is the same as the user we authorized as coffee00@flock
. Whenever you're able to get this sequence of requests working on your Zimbra server just input whatever you used for <WebDAV host>
into the Flock client and then we'll see where that takes us.
Thank you @rhodey for your clear and detailed explanation: it is really appreciated.
I tried to test http://webmail.web4all.fr/.well-known/carddav and it redirects with a 302 toward the https version of the same url. But the https version returns me a 411
HTTP/1.1 411 Length Required
Server: nginx
Date: Mon, 01 Dec 2014 22:15:03 GMT
Content-Type: text/html
Content-Length: 174
Set-Cookie: SERVERID=zose-proxy02a; path=/
<html>
<head><title>411 Length Required</title></head>
<body bgcolor="white">
<center><h1>411 Length Required</h1></center>
<hr><center>nginx</center>
</body>
</html>
Any suggestion on how to pursue the investigation?
can you copy + paste the exact curl command you ran for reference? minus the username & password credentials of course.
Of course, here it is:
curl -k -i -u username:password -X PROPFIND http://webmail.web4all.fr/.well-known/carddav --upload-file - -H "Depth: 0" <<end
<?xml version="1.0"?>
<a:propfind xmlns:a="DAV:">
<a:prop><a:current-user-principal/></a:prop>
</a:propfind>
end
The response being:
HTTP/1.1 302 Found
Cache-Control: no-cache
Content-length: 0
Location: https://webmail.web4all.fr/.well-known/carddav
Connection: close
But testing https://webmail.web4all.fr/.well-known/carddav returned a 411, like I said.
awesome, thanks!
I would expect curl to calculate the data length header itself and include this automatically but to check this try adding the -v
option and this will show us exactly what request you're sending. If you could include the output with the -v
option I think we'll be able to figure this 411 thing out.
Another possibility is that the Zimbra server is using some really weird SSL or TLS cipher suites, I've ran into this problem with curl before and adding the parameters to negotiate a more sane set of cipher suites was a pain.
Tested with -v
but unfortunately it didn't help
Here's the request
curl -v -k -i -u username:test -X PROPFIND https://webmail.web4all.fr/.well-known/carddav --upload-file - -H "Depth: 0" <<end
<?xml version="1.0"?>
<a:propfind xmlns:a="DAV:">
<a:prop><a:current-user-principal/></a:prop>
</a:propfind>
end
And here the complete response:
* Adding handle: conn: 0x7f9993003a00
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7f9993003a00) send_pipe: 1, recv_pipe: 0
* About to connect() to webmail.web4all.fr port 443 (#0)
* Trying 185.49.20.40...
* Connected to webmail.web4all.fr (185.49.20.40) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
* Server certificate: *.web4all.fr
* Server certificate: Gandi Standard SSL CA
* Server certificate: UTN-USERFirst-Hardware
* Server auth using Basic with user 'username'
> PROPFIND /.well-known/carddav HTTP/1.1
> Authorization: Basic dXNlcm5hbWU6dGVzdA==
> User-Agent: curl/7.30.0
> Host: webmail.web4all.fr
> Accept: */*
> Transfer-Encoding: chunked
> Depth: 0
> Expect: 100-continue
>
< HTTP/1.1 411 Length Required
HTTP/1.1 411 Length Required
* Server nginx is not blacklisted
< Server: nginx
Server: nginx
< Date: Mon, 01 Dec 2014 22:39:05 GMT
Date: Mon, 01 Dec 2014 22:39:05 GMT
< Content-Type: text/html
Content-Type: text/html
< Content-Length: 174
Content-Length: 174
< Set-Cookie: SERVERID=zose-proxy02a; path=/
Set-Cookie: SERVERID=zose-proxy02a; path=/
* HTTP error before end of send, stop sending
<
<html>
<head><title>411 Length Required</title></head>
<body bgcolor="white">
<center><h1>411 Length Required</h1></center>
<hr><center>nginx</center>
</body>
</html>
* Closing connection 0
I am asking my host if they know about Zimbra TLS cipher suites: I'll post here if I get anything.
I think these two lines in the HTTP request are our problem:
Transfer-Encoding: chunked
Expect: 100-continue
Our PROPFIND request is actually an XML document so technically we're sending a file to the WebDAV server, curl has decided that it wants to send this file chunked and after receiving an HTTP status 100 response from the server would then begin the transfer of the XML document. But as we can see the server does not return a 100 because for some reason I guess it doesn't feel like processing this request as chunked.
We need to tell curl to send this PROPFIND request in a single http request, then I expect the length will be included and the server will no longer give us a 411. Give me a few minutes to research how to do this in curl, maybe you'll find the answer too.
The Flock client doesn't do this chunked thing, ftr.
$ man curl
Remove an internal header by giving a replacement without content on the right side of the colon, as in: -H "Host:"
try adding a -H "Expect:"
to your curl command, this should remove the Expect: 100
header and hopefully cause curl to avoid sending this as chunked.
Nope :-/
Request:
curl -v -k -i -u user:password -X PROPFIND https://webmail.web4all.fr/.well-known/carddav --upload-file - -H "Depth: 0" -H "Expect:" <<end
<?xml version="1.0"?>
<a:propfind xmlns:a="DAV:">
<a:prop><a:current-user-principal/></a:prop>
</a:propfind>
end
Response
* Adding handle: conn: 0x7ff082803a00
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7ff082803a00) send_pipe: 1, recv_pipe: 0
* About to connect() to webmail.web4all.fr port 443 (#0)
* Trying 185.49.20.40...
* Connected to webmail.web4all.fr (185.49.20.40) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
* Server certificate: *.web4all.fr
* Server certificate: Gandi Standard SSL CA
* Server certificate: UTN-USERFirst-Hardware
* Server auth using Basic with user 'daniele%40pitrolo.it'
> PROPFIND /.well-known/carddav HTTP/1.1
> Authorization: Basic ZGFuaWVsZSU0MHBpdHJvbG8uaXQ6NGRlc3NvIHNvbm8gYSBjYXNB
> User-Agent: curl/7.30.0
> Host: webmail.web4all.fr
> Accept: */*
> Transfer-Encoding: chunked
> Depth: 0
>
< HTTP/1.1 411 Length Required
HTTP/1.1 411 Length Required
* Server nginx is not blacklisted
< Server: nginx
Server: nginx
< Date: Mon, 01 Dec 2014 23:17:23 GMT
Date: Mon, 01 Dec 2014 23:17:23 GMT
< Content-Type: text/html
Content-Type: text/html
< Content-Length: 174
Content-Length: 174
< Set-Cookie: SERVERID=zose-proxy02a; path=/
Set-Cookie: SERVERID=zose-proxy02a; path=/
<
<html>
<head><title>411 Length Required</title></head>
<body bgcolor="white">
<center><h1>411 Length Required</h1></center>
<hr><center>nginx</center>
</body>
</html>
got it :D, kinda... :|
curl -k -i -u <username>:<password> -X PROPFIND https://webmail.web4all.fr/.well-known/carddav --upload-file - -H "Depth: 0" -H "Content-Length: 105" -H "Transfer-Encoding:" -H "Expect:" <<end
<?xml version="1.0"?>
<a:propfind xmlns:a="DAV:">
<a:prop><a:current-user-principal/></a:prop>
</a:propfind>
end
HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Mon, 01 Dec 2014 23:28:51 GMT
Content-Length: 0
Location: http://webmail.web4all.fr:80/dav
Set-Cookie: SERVERID=zose-proxy02a; path=/
Cache-control: private
Weird that we get directed from an https resource to an http resource, but with that aside we should be able to execute our PROPFIND request on http://webmail.web4all.fr:80/dav
and finally get back the current-user-principal
property. but oddly enough this server gives us another redirect to https://webmail.web4all.fr/dav
, see below...
curl -k -i -u <username>:<password> -X PROPFIND http://webmail.web4all.fr:80/dav --upload-file - -H "Depth: 0" -H "Content-Length: 105" -H "Transfer-Encoding:" -H "Expect:" <<end
<?xml version="1.0"?>
<a:propfind xmlns:a="DAV:">
<a:prop><a:current-user-principal/></a:prop>
</a:propfind>
end
HTTP/1.1 302 Found
Cache-Control: no-cache
Content-length: 0
Location: https://webmail.web4all.fr/dav
Connection: close
only then after two redirects can we finally get the current-user-principal
property, like so...
curl -k -i -u <username>:<password> -X PROPFIND https://webmail.web4all.fr/dav --upload-file - -H "Depth: 0" -H "Content-Length: 105" -H "Transfer-Encoding:" -H "Expect:" <<end
<?xml version="1.0"?>
<a:propfind xmlns:a="DAV:">
<a:prop><a:current-user-principal/></a:prop>
</a:propfind>
end
except because I don't have a valid login on this server I of course get a 401 unauthorized.
HTTP/1.1 401 invalid username/password
Server: nginx
Date: Mon, 01 Dec 2014 23:40:07 GMT
Content-Length: 0
WWW-Authenticate: BASIC realm="Zimbra"
Set-Cookie: SERVERID=zose-proxy02a; path=/
In summary this zimbra server is configured kinda weird because we get redirected from an HTTPS resource to an HTTP resource and then back to HTTPS. The correct response to a PROPFIND on https://webmail.web4all.fr/.well-known/carddav
is a simple redirect to https://webmail.web4all.fr/dav
, this HTTP step in the middle is weird and unfortunately Flock will not follow nested redirects.
If you have any easy way to give your Zimbra server provider a heads up about this weird triple redirect that would probably be best although it is also possible that you could find an endpoint that does this redirect correctly, for example, like maybe https://webmail.web4all.fr/idk-why-this-works
would redirect directly to https://webmail.web4all.fr/dav
and skip this silly HTTP middle step.
Ok, the redirection towards http:/webmail.web4all.fr:80/dav/ confirms some tests someone else made. We have the whole picture now, and I hope this will help the provider fix this weird redirects.
Nonetheless, I tried
curl -k -i -u username:'password' -X PROPFIND https://webmail.web4all.fr/dav --upload-file - -H "Depth: 0" -H "Content-Length: 105" -H "Transfer-Encoding:" -H "Expect:" <<end
<?xml version="1.0"?>
<a:propfind xmlns:a="DAV:">
<a:prop><a:current-user-principal/></a:prop>
</a:propfind>
end
And I have been served a 400
:
HTTP/1.1 400 Bad Request
Server: nginx
Date: Mon, 01 Dec 2014 23:54:44 GMT
Content-Length: 0
Set-Cookie: SERVERID=zose-proxy02a; path=/
Since my password has blank spaces, I was worried that this might be the reason, and I tried twice, with simple quotes and without password (curl propts you to submit it then, which is perhaps better, since it won't show up in bash history). Seems like we aren't done yet, does it?
Fixing #73 you were right to underline that it wasn't sure Flock was going to support my Zimbra account! It took me a while but I was able to do a few tests and here is the result: Two logs actually, because I am unsure of whether I should sumbit as url $zimbraurl/dav or just $zimbraurl (first one) https://gist.github.com/anonymous/c6e40bc61a679f7628a0 https://gist.github.com/cbdb3beb9b77c134bef6
Thanks for looking at it when you get a little time!