CORS: Content Security headers block all data URIs

[happybeing]

Since the 0.5 API the Content Security headers have blocked loading of image resources by data URI and I think this is too strict. This is not something I understand well, but it seems ok to allow loading on non-script resources, as in:

X-Content-Security-Policy: allow 'self'; img-src data:

(See no. 3 at https://people.mozilla.org/~bsterne/content-security-policy/details.html)

[happybeing]

The above (img-src example) is for illustration only - I think it makes sense to lift this restriction for most things (image, text, binary), but as I say I don't really understand why block everything is the default. So I've decided to leave it to you guys!

[hitman401]

@theWebalyst, you are correct! Will make a note of this and make the amendment in the v0.6 which should soon follow.

[happybeing]

@krishnaIndia I've been playing with the CSP headers needed for the remoteStorage.js code to work. What I need appears to be:

    img-src \'self\' data:; 
    form-action \'self\' http://api.safenet;
    style-src \'self\' \'unsafe-inline\';

The form-action seems only necessary to avoid a console error, but not necessary for the code to work.

The style-src 'unsafe-inline' is necessary for a widget that displays the connection buttons to allow an application to connect to a server (i.e. SAFE Launcher) but obviously may not be suitable in the long term.

Is it possible you can allow the 'unsafe-inline' at least temporarily while I see if there's an alternative way for remoteStorage.js to do what it needs without this? I've raised it with them.

[hitman401]

@theWebalyst, the CSP is being enforced from the proxy. As you are aware, the safe_browser should help in removing the proxy from launcher soon.

You can temporarily suppress the CSP validations by starting the launcher locally using npm run start-dev instead of npm start. If you are running the launcher binary directly, then start the executable by passing the --proxy_unsafe_mode flag. Example, safe_launcher --proxy_unsafe_mode

Based on the forum thread next on the cards wil be the API integration with the browser. We can remove the proxy from the launcher once this feature is integrated in the safe browser.

[happybeing]

Thanks @krishnaIndia. The penny dropped on the separation of the web proxy yesterday when I realised why I didn't see web_proxy.js in the debugger source listing.. Doh! :-) Thanks for the unsafe stuff. I noticed this in the web proxy code but hadn't figured out how to use it. That gives me a way to share the code for people to try out, though it will no doubt confuse users if they have to start safe_launcher differently to have these apps work. So it might still be worth loosening it while the proxy lives on?

I take your point about SAFEr Browser, but this also raises questions for me:

1) One is whether SAFEr Browser should by default impose its own CSP? In the absence of a web server website can insert its own CSP headers in HTML, but this is not as effective and offers less protection. I wonder if @joshuef has any thoughts on this!

2) It also leads me to wonder what is anticipated regarding legacy browsers. Do we assume that there will be no encouragement or development of solutions for Firefox and Chrome for example? In which case why are we sticking with ".safenet"? That leads me to suppose we still want plugins for those, but that they will either require a companion web proxy, or some other solution that I'm not clear about. Can you or @Viv-Rajkumar clarify current thinking on this and, and if you have any expectations about CSP restrictions (particularly style-src 'unsafe-inline') that might be blockers for me with the current remoteStorage.js?

Sorry to raise so many questions but if we can tie any of this down now it could save wasted effort and give me/RS.js team time to consider ways to jump these hurdles. We're blazing a trail here :-)

[happybeing]

@krishnaIndia I tried safe_launcher --proxy_unsafe_mode with the alpha1 and it stops the proxy working at all. Without it I can access dir.yvette.safenet etc but with it I just get 404 / Invalid Endpoint.

Is this a bug, and if not can we have a solution that works with the next alpha release (either --proxy_unsafe_mode, or preferably looser CSP headers)? :-)

[hitman401]

1) One is whether SAFEr Browser should by default impose its own CSP? In the absence of a web server website can insert its own CSP headers in HTML, but this is not as effective and offers less protection. I wonder if @joshuef has any thoughts on this!

I don't think CSP would be added in the SAFE Browser. @josheuf can confirm if I am wrong.

2) It also leads me to wonder what is anticipated regarding legacy browsers. Do we assume that there will be no encouragement or development of solutions for Firefox and Chrome for example?

We can support via extensions is what I can guess. Firefox and chrome provides options to inject javascript in every window through an extension. And safe sites will have to use http on legacy browsers. It should be possible to spawn a small server and just do what the proxy was doing. This would be more an inbuilt proxy within an extension. A chrome sample is here for building a server. Similar implementation can also be done in firefox too.

This is just one approach I can quickly think. This might have its own limitations and we can also figure out better means if any to serve the safe sites on chrome and firefox.

I would suggest to use an idiomatic api in you code and start the launcher in unsafe_proxy_mode (Since CSP is a blocker in your case). This will make it really easy for you to port your code to be compatible with SAFE Browser.

The --proxy_unsafe_mode option should be working fine. I had suggested the same for @lightyear in an issue previously. Can you try running npm run start-dev, if you were building launcher locally?

[happybeing]

Thanks again @krishnaIndia this is very helpful.

I would like a way to have these remoteStorage.js based apps available for others to use on the alpha network, and unless --proxy_unsafe_mode works we still can't do this (and even then its not ideal as people who visit the URLs without knowing this will get errors). npm run start-dev doesn't really help because I can modify the web_proxy.js when working locally anyway. So I need a way of this working with the standard launcher which means either a temp change to the next alpha release (which should be ok short term I would think), getting --proxy_unsafe_mode fixed, or me finding a way to modify remoteStorage.js which may be tricky.

I'll check with @ligthyear how he got on with --proxy_unsafe_mode but it doesn't seem to work with alpha1 launcher (how long ago was it you suggested it to him?).

[frabrunelle]

Here's what I see when I try to access a .safenet website after starting SAFE Launcher v0.8.0 (Alpha 1) with --proxy_unsafe_mode:

INFO 18:21:26.820 - Starting proxy server
DEBUG 18:21:26.919 - Proxy Server - onMessage event - received -
INFO 18:21:26.919 - Proxy server started
DEBUG 18:21:26.922 - Proxy Server - onMessage event - received -
INFO 18:21:26.922 - Proxy started in unsafe mode
DEBUG 18:21:44.566 - Unauthorised request ::/
DEBUG 18:21:44.600 - Unauthorised request ::/favicon.ico
DEBUG 18:22:27.467 - Invoking CLIENT STATS API:: GETs COUNT::client-stats
DEBUG 18:22:27.467 - Sending message to FFI - 0dcc3f5d-c58a-4f15-ae5d-aff8cd1ff84e - client-stats - gets
DEBUG 18:22:27.468 - FFI response code 0 for id - 0dcc3f5d-c58a-4f15-ae5d-aff8cd1ff84e
VERBOSE 18:22:27.469 - Invoking response callback :: callback id - 0dcc3f5d-c58a-4f15-ae5d-aff8cd1ff84e
DEBUG 18:22:27.478 - Unauthorised request ::/
DEBUG 18:22:30.62 - Unauthorised request ::/
DEBUG 18:22:30.112 - Unauthorised request ::/favicon.ico
DEBUG 18:22:30.119 - Unauthorised request ::/favicon.ico
DEBUG 18:22:57.541 - Unauthorised request ::/

And in the browser, here's the error I see:

{"errorCode":404,"description":"Endpoint Not Found"}

[happybeing]

I spoke to @ligthyear and he thinks he used

npm run start-dev (which internally runs node ./tasks/start --proxy_unsafe_mode)

rather than safe_launcher --proxy_unsafe_mode

Regardless safe_launcher --proxy_unsafe_mode doesn't work as above with the alpha1.

UPDATE: Problem seems to be that --proxy_unsafe_mode is not just disabling the insertion of CSP headers, it looks like it assumes a dev environment (localhost) and skips re-direction of www amongst other things. See: https://github.com/maidsafe/safe_launcher/blob/master/app/server/web_proxy.js#L68-L70 It's definitely not suitable for my purposes, which is to allow a normal user to run without the CSP injection.

UPDATE: I tried SAFE Beaker Browser (see dev forum post) and it too hits the CORS/CSP issue for sites served from the alpha1 network, and to me it makes sense that it should implement just like Chrome/FF. I'm not sure why you think it wouldn't @krishnaIndia? Obviously when it accesses the API through an injection i/f the problem will go away, but for now I think it is the same with all browsers so we come back to my request for a temporary solution (until we can just say "use SAFE Beaker") with the next alpha network/launcher (a command line switch would be fine so long as it works on the binary distribution).

UPDATE: Can we have a clean way of disabling CSP headers that would work for normal users on alpha (non development)? That would be my best solution for the time being while we work out what to do in the proxy and at my end, in remoteStorage.js.

[happybeing]

@krishnaIndia Looks like remoteStorage.js will be updated soon to avoid the need for style-src 'unsafe-inline' so my request is now just for the following in v0.6 Launcher proxy CSP:

img-src 'self' data:; 
form-action 'self' http://api.safenet;

With that my RS.js apps should be ready to go before you release v0.6 ... race ya! :smile:

[joshuef]

@theWebalyst , @krishnaIndia is right, I don't think there's a place/need for CSP adjustment in the SAFE Browser, it would then be altering the response the launcher provides, which would seem redundant as far as SAFE core stuff goes.

I think if there's a good reason for CSP changes then we would want that reflected in the launcher anyway (and I think what you're proposing here seems reasonable and an important change).

So yeh, @theWebalyst any launcher CSP things you hit will also be hit in the SAFE Browser, as such we're limited currently.

Going forward safe-js integration should get around CORS/CSP for certain types of API access (as you wont really be doing ajax requests etc, just accessing window.safe functions), although standard in-page content loading will still be subject to launcher.

---

To me this is a bigger question of what actually do we want to allow? Mixed content (ie. http/s access etc) while on safe: sites or not? These things should, I think, be consistent across the core implementations (ie. simply served from the launcher and its CSP policy).

(And in fact, if we are strict about this, then the SAFE Browser would actually not need any 'clearnet' switch, as by way of using the safe: protocol, the CSP would/could prevent any mixed content loading. This is probably a Q to follow up on the forums. )

[happybeing]

@krishnaIndia I see 0.9.0 is unchanged in relation to web_proxy CSP. Do you have an expectation around resolving this, or if not at least allowing in an upcoming version. It would help by making it easier for me to share stuff without hacking RS.js each time:

img-src 'self' data:; 
form-action 'self' http://api.safenet;