Ensure that Thunderbird can access cyrus-imapd CalDAV through a reverse proxy such as nginx.

As a user of Thunderbird (and thus Thunderbird's Lightning extension), I want to be able to use cyrus-imapd CalDAV as my calendar server.

How to reproduce

Setup an nginx HTTPS server with a cyrus-imapd CalDAV server as upstream server.
Run curl --verbose -u user1@example.com:"$PASSWORD" --basic -i -X PROPFIND -H 'Depth: 1' https://calendar.example.com/dav/calendars/user/user1@example.com/Default/.
Observe that this request is successful.
Run curl --verbose -u user1@example.com:"$PASSWORD" --digest -i -X PROPFIND -H 'Depth: 1' https://calendar.example.com/dav/calendars/user/user1@example.com/Default/.

Expected result

Curl sends a request, and cyrus-imapd CalDAV denies this request with a digest authentication challenge.
Curl sends a another request with a digest authentication response, and cyrus-imapd CalDAV accepts this request.

Actual result

Curl sends a request, and cyrus-imapd CalDAV denies this request, replying with a digest authentication challenge.
Curl sends a another request with a digest authentication response, and cyrus-imapd CalDAV denies this request, replying with a digest authentication challenge.

Consequence

Using Thunderbird (version 78 or later) or Lightning (which, by default, require HTTP Digest authentication) as a client to cyrus-imapd CalDAV is unnecessarily impossible.

Workaround

When the connection between the nginx server and the upstream cyrus-imapd CalDAV server is configured by the nginx side to be kept alive (no "Connection: close" header is sent by nginx and the protocol version in the request sent by nginx is HTTP/1.1 instead of HTTP/1.0) and there is low traffic, then the problem disappears in many cases.

Analysis

The cyrus-imapd httpd code implicitly assumes that the second request issued by the HTTP client will reach the same "httpd" process in the same TCP connection directly after the first request issued by the same HTTP client. This assumption is unwarranted, as HTTP is defined to be stateless. It is wrong in many cases, for example when there is a reverse proxy between the HTTP client and the HTTP server, and the reverse proxy closes the TCP connection to the HTTP server after each request (which is the default for nginx).

This implicit assumption is expressed in the current code in imap/httpd.c, as it assumes that it is permissible to store state in the global variable

    static int status = SASL_OK;

which gets initialized only once and then overwritten by

            status = sasl_server_start(httpd_saslconn, scheme->saslmech,
                                       clientin, clientinlen,
                                       &serverout, &serveroutlen);

which may (or may not) switch the value of status from SASL_OK to SASL_CONTINUE.

This switched value then leaks to the next request processed by the same "httpd" process. If the next request happens to exist at all (which may not be the case if the HTTP proxy or the HTTP client closes the TCP connection in the meantime) and happens to come from the same client (which may not be the case as the next request from the same client may get routed to a different "httpd" process on the same or a different machine or as an unrelated HTTP request is interleaved by the HTTP proxy between the 2 requests from the same HTTP client), then the digest authentication may actually work. But if not, then the digest authentication fails.

The solution is to get rid of the static-ness of the "status" variable while still supporting digest authentication. Then the digest authentication support would not be unreliable anymore, as every HTTP request would be treated equally.

cyrusimap / cyrus-imapd