CORS errors on Firefox 73 when using CourtListener's APIs

[mlissner]

Firefox 71 is not affected, I haven't checked Firefox 72, but in Firefox 73 (currently nightly) you get the following errors when trying to use the RECAP extension:

RECAP: Getting availability of docket 178502 at dcd recap.js:29:15
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://www.courtlistener.com/api/rest/v3/dockets/?pacer_cas…2C11%2C13%2C15&court=dcd&fields=absolute_url%2Cdate_modified. (Reason: CORS header ‘Access-Control-Allow-Origin’ does not match ‘null’).

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://www.courtlistener.com/api/rest/v3/dockets/?pacer_cas…2C11%2C13%2C15&court=dcd&fields=absolute_url%2Cdate_modified. (Reason: CORS request did not succeed).

RECAP: Ajax error getting docket availability. Status: error. Error: .

I'm not sure if this should be fixed by CourtListener's server or by a fix in the extension. This feels a lot like #280, which seems fine at the moment, but portends badly.

I discovered this while investigating this blog post about their changes to content_security_policy:

https://blog.mozilla.org/addons/2019/12/12/test-the-new-csp-for-content-scripts/

I don't think we can test that though until this is fixed.

[Rob--W]

(via https://mail.mozilla.org/pipermail/dev-addons/2019-December/004243.html )

Could you run mozregression to find when the regression occurred? I cannot reproduce the bug because I don't have credentials to log in to the site.

[mlissner]

OK, so I followed up a bunch on this today. Mozilla seems a little unclear on what's happening here. They've got a few different approaches they've tried, and they've landed then backed out changes several times. I'm lost as to what the design is and what the status is, so I asked on the mailing list. Hopefully we'll get some clarity there (they've been good about responding to messages so far).

I just tested this in both 72 and 74 and it's working fine in both. I also had to do some research to refresh my memory about CORS (which I hadn't looked at in many years). The basic idea is that certain requests from the browser make "pre-flight" requests to ask the server what's allowed. They do this using OPTION requests like this:

↪ http OPTIONS https://www.courtlistener.com/api/rest/v3/ Origin:google.com --headers 
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: accept, accept-encoding, authorization, content-type, dnt, origin, user-agent, x-csrftoken, x-requested-with
Access-Control-Allow-Methods: GET, HEAD, OPTIONS
Access-Control-Allow-Origin: google.com
Access-Control-Max-Age: 86400
Allow: GET, HEAD, OPTIONS
Connection: Keep-Alive
Content-Length: 248
Content-Type: application/json
Date: Tue, 14 Jan 2020 01:02:50 GMT
Keep-Alive: timeout=5, max=100
Server: Apache/2.4.7 (Ubuntu)
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Vary: Accept,Origin,Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-UA-Compatible: IE=Edge,chrome=1
X-XSS-Protection: 1; mode=block

The change Mozilla is thinking about making here is to change origin to null, like this, I think:

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: accept, accept-encoding, authorization, content-type, dnt, origin, user-agent, x-csrftoken, x-requested-with
Access-Control-Allow-Methods: GET, HEAD, OPTIONS
Access-Control-Allow-Origin: null
Access-Control-Max-Age: 86400
Allow: GET, HEAD, OPTIONS
Connection: Keep-Alive
Content-Length: 248
Content-Type: application/json
Date: Tue, 14 Jan 2020 01:03:02 GMT
Keep-Alive: timeout=5, max=100
Server: Apache/2.4.7 (Ubuntu)
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Vary: Accept,Origin,Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-UA-Compatible: IE=Edge,chrome=1
X-XSS-Protection: 1; mode=block

I don't fully understand why that breaks things for us. On the server, we allow CORS requests to the API from literally anywhere:

# CORS
CORS_ORIGIN_ALLOW_ALL = True
CORS_URLS_REGEX = r"^/api/.*$"
CORS_ALLOW_METHODS = (
    "GET",
    "HEAD",
    "OPTIONS",
)
CORS_ALLOW_CREDENTIALS = True

Hopefully we can close this bug when Mozilla fixes things on their end. I'm really unsure who has the issue at this point.

[mlissner]

Whelp, much ado about nothing I guess, but hey, bonus, I got a re-education about CORS and now I have Firefox, Firefox-Next, and Firefox-Trunk on my computer.

I just heard back on the list that Mozilla has backed out their change for this due to too much breakage. You can see some of that here:

https://bugzilla.mozilla.org/show_bug.cgi?id=1607154

I just tested on 72, 73, and 74, and I can't reproduce this anymore. For the moment at least, this is fixed. I'm not yet sure if they plan to try another approach (I asked on the list) or if this is the end of this issue. We'll find out soon enough, I suppose.