Unable to import AiCore library due to CORs

[justyns]

Access to fetch at 'https://ai.silverbullet.md/index.json' from origin 'https://sb.lan.domain' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

I get the above error when trying to import ai.silverbullet.md/Library/AICore. Supposedly github pages allows all origins:

❯ curl -v https://ai.silverbullet.md/index.json -o /dev/null 2>&1| grep -i access
< access-control-allow-origin: *

I'm wondering if it's the preflight request causing an issue though because I do get a 405 with any OPTIONS request:

❯ curl -X OPTIONS -v https://ai.silverbullet.md/index.json -o /dev/null 2>&1
...
< HTTP/2 405

[justyns]

I learned something new about CORs today. From https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS

Some requests don't trigger a CORS preflight. Those are called simple requests from the obsolete CORS spec, though the Fetch spec (which now defines CORS) doesn't use that term. ... Apart from the headers automatically set by the user agent (for example, Connection, User-Agent, or the other headers defined in the Fetch spec as a forbidden header name), the only headers which are allowed to be manually set are those which the Fetch spec defines as a CORS-safelisted request-header, which are: Accept Accept-Language Content-Language Content-Type (please note the additional requirements below) Range (only with a simple range header value; e.g., bytes=256- or bytes=127-255)

So our federation requests aren't considered "simple" due to having extra headers (X-Sync-Mode, X-Get-Meta). And github pages doesn't support preflight requests.

@zefhemel do you have a preference on how to fix this? I can open a PR, but had two ideas:

1) remove the X-Get-Meta and X-Sync-Mode headers from the federation fetch calls OR 2) set mode: no-cors on the federation fetch calls

edit: and the 3rd option I guess would be to not use gh pages

I'm honestly not sure what the "correct" solution is, but I tested both of these options and they seem to work fine in my limited testing.

[zefhemel]

Right, I feared this may be a problem. On silverbullet.md I solve it by adding CORS headers, but I don't think you can do that on gh-pages.

I've played with various options and ended up back at the current headers and trying to recall why. The issue with not sending X-Sync-Mode is that an actual silverbullet server (if you were to federate with it), such as one in read-only mode (e.g. silverbullet.md), will send you a redirect to the web (index.html) page without it. I think I tried figuring out a way to detect that somebody made a CORS call to it, and then not do the redirect but not sure that worked. Looking at http_server.ts around line 489 I see some funky code that may work but from my memory not reliably and perhaps not across all browsers. So to be checked here: does removing the X-Sync-Mode still let federating with another SB instance work? From what I remember it also seemed to matter if you had a tab open on that federated server before, so perhaps caching had an impact somehow 🤷🏻

The X-Get-Meta header can in principle be removed, but what will happen that when looking at a federate page, it will pull down the entire content every 5s or so, throwing away the content, because it's only interested in the headers. Of course the proper thing to do here is to use a HEAD request instead of a GET, but from the comments in http_space_primitives.ts:

This used to use HEAD, but it seems that Safari on iOS is blocking cookies/credentials to be sent along with HEAD requests

[justyns]

Thanks for the explanation, it gives me some things to look at.

I'd rather not break federation with actual SB servers, but I also think being able to "federate" with static sites like gh pages would be good. The problem with gh pages specifically is that the OPTIONS preflight check just errors out entirely. If you do a normal GET request, it does return cors headers.

Maybe something like this would work?

• Use nativeFetch() with X-Sync-Mode / X-Get-Meta headers
• If the fetch fails (assuming we can detect the error was during a preflight check), make a second request, this time without the extra headers

Or some way to identify a real SB server vs something else, like requesting index.json once with no special headers and have some specific key in index.json saying whether its a real server. It could include options for how often to poll for changes/etc too

[justyns]

So a completely different idea.. Does federation (and library imports) need to happen on the client side at all? Could it be moved to server-side only?

For federation, if I'm running a server that has a bunch of people federating pages from me, I'd rather only their server be reaching out for updates vs their server and all of their individual clients too.

Is it a goal to have the client be 100% independent, or is it expected to always be able to occasionally check in with the server and just support periods of being offline?

[justyns]

This was fixed in the latest Silverbullet (0.8.x, thanks @zefhemel ). Importing libraries now happens on the server, and we can import the AICore library now too.

I updated the doc at https://ai.silverbullet.md/AI%20Core%20Library/ with the new instructions and its working as expected for me.