Support CORS requests with the JSON API

[pyrmont]

Issue: Micro.blog's JSON API does not support cross-origin resource sharing (CORS) requests. This makes developing a browser-based JavaScript app impossible.

Solution: Enable CORS requests by:

1. supporting preflight requests from browsers;
2. and either:
   1. whitelisting particular domains; or
   2. allowing the JSON API token to be passed via a custom header (e.g. X-MICROBLOG-TOKEN).

Rationale: Micro.blog provides a JSON API that any developer can use to interface with the Micro.blog service. For reasons explained in more detail below, JavaScript running in a browser cannot use this API outside of simple GET requests.

Support of the open web is a foundational value of Micro.blog. Allowing users to use browser-based JavaScript apps is consistent with this value and encourages the use of the web as platform for delivering rich applications.

Use Case: I am currently developing a Micro.blog client as a JavaScript single-page application using the React framework.* I want the client to support all of the functionality available via the Micro.blog JSON API.

* I'm developing the application using ClojureScript but it will be compiled to JavaScript.

Unfortunately, CORS POST requests that set the value of the Content-Type to application/json will fail to be sent unless the target server (i.e. Micro.blog's server) supports CORS requests.

Technical Details: Except for simple requests (e.g. GET requests), modern browsers will generally not perform XMLHttpRequest calls if the domain of the resource differs from the domain on which the JavaScript file has been served. This is 'cross-origin resource sharing' (CORS).

A browser will perform the call if it can successfully conduct a preflight test with the target server and confirm that the server supports CORS requests. It does this by making a request using an OPTIONS request and querying the capabilities of the server.

The first step to supporting CORS requests is for the Micro.blog server to respond to these OPTIONS requests. Micro.blog's JSON API is served via Nginx and a simple commented configuration file for that type of server is available here.

The second step is to decide how to permit clients to submit CORS requests. While certain CORS requests can be accepted from any origin by setting the Access-Control-Allow-Origin to the wildcard * this does not work if the request includes an Authorization header. Currently, Micro.blog requires requests to the API include the authorisation token in the Authorization header.

There are two ways to solve this problem. The first is by whitelisting the domains from which requests can originate. The app I am developing with eventually be hosted at github.io and so one solution would be to list that domain in the Access-Control-Allow-Origin header returned in preflight.

While this solution is simple, it would make development difficult as the development server will not be github.io. Perhaps more importantly, it introduces a gatekeeper to the use of the Micro.blog API that other clients do not need to worry about.

The second solution is to accept the authentication token in a custom header. Micro.blog could continue to accept the token in the Authorization header but would also allow a client to submit it using a custom header (e.g. X-MICROBLOG-TOKEN). This would allow the server to use the * wildcard while ensuring that clients are authenticated.

Additional details:

• https://medium.com/@baphemot/understanding-cors-18ad6b478e2b
• https://www.html5rocks.com/tutorials/cors/

[ghost]

cc: @adamprocter

[adamprocter]

Nice. I hit the same issue with my Vue app a while back @manton is aware this is a block to developing apps. Thanks @simon-woods for the heads up. https://microcard.adamprocter.co.uk/#/

[pyrmont]

Just noting the following:

• @manton is indeed aware of the issue and needs to make a change to how MB handles authentication;
• CORS and browser-based clients were discussed on the MB Slack in approximately March 2019.

[pyrmont]

For my own reference more than anything else, this test with Curl of a preflight request (source) should return successfully when this is set up:

curl -H "Origin: http://example.com" -H "Access-Control-Request-Method: POST" -H "Access-Control-Request-Headers: X-Requested-With" -X OPTIONS --verbose https://micro.blog/posts/favorites

Currently the response is 404.

[manton]

I've been making some under-the-hood changes to support this. It should be ready fairly soon. Thanks!

[staff0rd]

Any updates this? I'm guessing it's related to an issue I'm seeing specific to CORS but for feed.xml.

I'm making a call to https://stafford.micro.blog/feed.xml from staffordwilliams.com. This call works on Chrome but fails on Safari due to a CORS preflight check.

My not-amazing work-around for now is to prefix the the address with https://cors-anywhere.herokuapp.com/.

[manton]

@staff0rd I wonder if another work-around for now would be to use a custom subdomain for your microblog, e.g. micro.staffordwilliams.com? I think web browsers might allow the request in that case. (But I might be wrong. I haven't checked the CORS rules for subdomains.)

[adamprocter]

My stuff was on a sub domain which I use for M.B. and didn’t work

discursive.adamprocter.co.uk microcard.adamprocter.co.uk

[staff0rd]

It appears cors-anywhere needs to explicitly limit even low-volume traffic by the end of this month. There are a number of security concerns hosting your own instance also.

The approach mentioned at the top of this thread of

setting the Access-Control-Allow-Origin [header] to the wildcard *

would solve for feed.xml but I agree it wouldn't solve for the JSON API. Any chance of allowing cross-origin requests for RSS @manton? Could split this out to another issue as to not derail the JSON API issue...

[staff0rd]

@adamprocter is correct re: subdomains, i've aliased stafford.micro.blog as devlog.staffordwilliams.com, but, my XHR call is from staffordwilliams.com, not the subdomain.

I can confirm that making a GET request to the rss feed in Chrome works as expected, but in Safari it does not. It appears some CORS requests are classified as Simple Requests. These do not require the preflight (OPTIONS) check. It also appears that Chrome doesn't make a preflight check but that Safari does:

curl -H "Origin: http://example.com" https://stafford.micro.blog/feed.xml

works as expected and is what Chrome executes (no preflight).

curl -H "Origin: http://example.com" -X OPTIONS https://stafford.micro.blog/feed.xml

Fails with 405 Not Allowed and represents the preflight check that Safari performs.

[staff0rd]

I was able to resolve my particular issue with Safari by using the following Cloudflare worker;

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})

async function handleRequest(request) {
  let response = await fetch(request)
  let newHeaders = new Headers(response.headers)
  newHeaders.set("Access-Control-Allow-Origin", "*")

  if (request.method === "OPTIONS") {
    newHeaders.set("Access-Control-Allow-Headers", "User-Agent");
    return new Response(response.body, {
      status: 200,
      statusText: 'OK',
      headers: newHeaders
    })
  }

  return new Response(response.body, {
    status: response.status,
    statusText: response.statusText,
    headers: newHeaders
  })
}

[tomcritchlow]

Hey just checking in here - seems like this is still an issue?

I would like to embed my microblog posts on my site - I tried sidebar.js but it only loads truncated posts, not full HTML contents of each post? That's not going to work for me so I'm trying to use the /feed.json to fetch the full post HTML for latest posts but running into the CORS issue here...

[manton]

Sorry, this is still an issue. I'll take another look today and see if I can update this. (Long story short, the feeds for hosted blogs are handled by separate servers than the rest of Micro.blog's JSON API, so need to be configured separately.)

[manton]

Update: I've added access-control-allow-origin to hosted blogs. Let me know if that fixes it.

[tomcritchlow]

Awesome thank you!

[pyrmont]

I'm no longer developing the client but assume from @tomcritchlow's response that this has fixed the issue and will mark this as closed.