Backend Security Checklist

[jvehent]

Risk Management

• [x] The service must have performed a Rapid Risk Assessment and have a Risk Record bug
• [x] Public staging and production endpoints must be added to the security baseline

Infrastructure (for ops)

• [x] Access and application logs must be archived for a minimum of 90 days
• [x] Use Modern or Intermediate TLS
• [x] Set HSTS to 31536000 (1 year)
  • strict-transport-security: max-age=31536000
• [x] Set HPKP to 5184000 (60 days)
  • Public-Key-Pins: max-age=5184000; pin-sha256="WoiWRyIOVNa9ihaBciRSC7XHjliYS9VwUGOIud4PB18="; pin-sha256="r/mIkG3eEpVdm+u/ko/cwxzOMo1bk4TyHIlByibiA5E="; pin-sha256="YLh1dUR9y6Kja30RrAn7JKnbQG/uEtLMkBgFF2Fuihg="; pin-sha256="sRHdihwgkaib1P1gxX8HFszlD+7/gTfNvuAybgLPNis=";
  • Start with max-age set to 5 minutes (max-age=300) and increase progressively
  • The first two pins are for Digicert EV and DV roots, the last two are for Let's Encrypt X3 and X4 intermediates (LE is only used for backup)
  • [x] If the service is not hosted under services.mozilla.com, it must be manually added to Firefox's preloaded pins.
• If service has an admin panels, it must:
  • [x] only be available behind Mozilla VPN (which provides MFA)
  • [x] require Auth0 authentication

Development

• [x] Sign all release tags, and ideally commits as well
  • Developers should configure git to sign all tags and upload their PGP fingerprint to https://login.mozilla.com
  • The signature verification will eventually become a requirement to shipping a release to staging & prod: the tag being deployed in the pipeline must have a matching tag in git signed by a project owner. This control is designed to reduce the risk of a 3rd party GitHub integration from compromising our source code.
• [x] Keep 3rd-party libraries up to date
  • Use NSP or [GreenKeeper](https://greenkeeper.io/ Greenkeeper) for NodeJS applications
  • For Python applications, enable pyup security updates:
  • Add a pyup config to your repo (example config: https://github.com/mozilla-services/antenna/blob/master/.pyup.yml)
  • From the "add a team" dropdown for your repo add the relevant "Approved Mozilla PyUp Configuration" team for your github org (e.g. for mozilla and mozilla-services) and grant it write permission.
  • Notify secops@mozilla.com to enable the integration in pyup
  • Consider using pip list --outdated or requires.io too
• [x] Integrate static code analysis in CI, and avoid merging code with issues
  • Javascript applications should use ESLint with the Mozilla ruleset
  • Python applications should use Bandit
  • Go applications should use the Go Meta Linter
  • Use whitelisting mechanisms in these tools to deal with false positives

Dual Sign Off

• [x] Services that push data to Firefox clients must require a dual sign off on every change, implemented in their admin panels
  • This mechanism must be reviewed and approved by the Firefox Operations Security team before being enabled in production

Logging

• [x] Publish detailed logs in mozlog format (APP-MOZLOG)
  • Business logic must be logged with app specific codes (see FxA)
  • Access control failures must be logged at WARN level

Security Headers

• [x] Must have a CSP with
  • [x] a report-uri pointing to the service's own /__cspreport__ endpoint
  • [x] web API responses should return default-src 'none'; frame-ancestors 'none'; base-uri 'none'; report-uri /__cspreport__ to disallowing all content rendering, framing, and report violations
  • [x] if default-src is not none, frame-src, and object-src should be none or only allow specific origins
  • [x] no use of unsafe-inline or unsafe-eval in script-src, style-src, and img-src
• [x] Web APIs must set a non-HTML content-type on all responses, including 300s, 400s and 500s
• [x] Set the Secure and HTTPOnly flags on Cookies, and use sensible Expiration
• [x] Make sure your application gets an A+ on the Mozilla Observatory
• [x] Verify your application doesn't have any failures on the Security Baseline.
  • Contact secops@ or ping 'psiinon' on github to document exceptions to the baseline, mark csrf exempt forms, etc.
• [x] Web APIs should export an OpenAPI (Swagger) to facilitate automated vulnerability tests

Security Features

• [x] Authentication of end-users should be via FxA. Authentication of Mozillians should be via Auth0/SSO. Any exceptions must be approved by the security team.
• [x] Session Management should be via existing and well regarded frameworks. In all cases you should contact the security team for a design and implementation review
  • Store session keys server side (typically in a db) so that they can be revoked immediately.
  • Session keys must be changed on login to prevent session fixation attacks.
  • Session cookies must have HttpOnly and Secure flags set.
  • For more information about potential pitfalls see the OWASP Session Management Cheet Sheet
• [x] Access Control should be via existing and well regarded frameworks. If you really do need to roll your own then contact the security team for a design and implementation review.

Databases

• [x] All SQL queries must be parameterized, not concatenated
• [x] Applications must use accounts with limited GRANTS when connecting to databases
  • In particular, applications must not use admin or owner accounts, to decrease the impact of a sql injection vulnerability.

Common issues

• [x] User data must be escaped for the right context prior to reflecting it
  • When inserting user generated html into an html context:
  • Python applications should use Bleach
  • Javascript applications should use DOMPurify
• [x] Apply sensible limits to user inputs, see input validation
  • POST body size should be small (<500kB) unless explicitely needed
• [x] When managing permissions, make sure access controls are enforced server-side
• [x] If handling cryptographic keys, must have a mechanism to handle quarterly key rotations
  • Keys used to sign sessions don't need a rotation mechanism if destroying all sessions is acceptable in case of emergency.
• [x] Do not proxy requests from users without strong limitations and filtering (see Pocket UserData vulnerability). Don't proxy requests to link local, loopback, or private networks or DNS that resolves to addresses in those ranges (i.e. 169.254.0.0/16, 127.0.0.0/8, 10.0.0.0/8, 100.64.0.0/10, 172.16.0.0/12, 192.168.0.0/16, 198.18.0.0/15).

[jvehent]

I crossed off a bunch of items since this is a static site. Please go through the rest before going live.

[jbuck]

@jvehent two quick q's about the checklist:

Access and application logs must be archived for a minimum of 90 days

So that'd mean configuring the Cloudfront logs and storing them for 90 days, correct?

a report-uri pointing to the service's own /cspreport endpoint

Is there a csp reporter that someone has configured for a static site before? I think it'd be possible to do with Lambda Edge if someone hasn't figured this out already

[jvehent]

So that'd mean configuring the Cloudfront logs and storing them for 90 days, correct?

Yep, that's correct.

Is there a csp reporter that someone has configured for a static site before?

Not that I'm aware. I wonder if we want to relax this requirement for sites that don't handle authentication, which would make your life easier. @psiinon, any thoughts?

[jbuck]

Two more clarifications needed :)

If the service is not hosted under services.mozilla.com, it must be manually added to Firefox's preloaded pins.

How do I go about adding themer.firefox.com to Firefox's preloaded pins?

no use of unsafe-inline or unsafe-eval in script-src, style-src, and img-src

I had to enable unsafe-inline in style-src. @lmorchard do you know if it's possible to not set inline styles? Given the extensive uses of colour pickers, I can see how it'd be difficult to avoid setting inline styles.

[lmorchard]

I had to enable unsafe-inline in style-src. @lmorchard do you know if it's possible to not set inline styles? Given the extensive uses of colour pickers, I can see how it'd be difficult to avoid setting inline styles.

Yeah, I think this project is pretty reliant on inline styles for the theme previews that involve arbitrary user-selected colors. I can scratch my head on it for a bit, but I'm fairly sure we can't get around that one

[jvehent]

How do I go about adding themer.firefox.com to Firefox's preloaded pins?

I think we'll skip that in this instance. It's a code change in Firefox (hard to do) and themer is only a testpilot experiment, so the risk is low.

Yeah, I think this project is pretty reliant on inline styles for the theme previews that involve arbitrary user-selected colors. I can scratch my head on it for a bit, but I'm fairly sure we can't get around that one

Allow me to cast an @april spell on this CSP, and with a bit of luck, she'll help find a solution here.

[april]

In general I am okay with the use of 'unsafe-inline' inside style-src. There are a few anti-patterns that you should watch out for though:

• Don't use Javascript frameworks to set value on form entry values if the user is allowed to define CSS
• There are some weird attacks around iframes and fonts, which you can mitigate by being careful with where frames are allowed

Other than that, your big concern is phishing, which is annoying but not the end of the world. I will add that the CSP3 working group is considering proposals to add something like 'unsafe-inline-attr' where style= would be allowed but not <style> tags. That's a ways off at best though.

There are some other ways that might be more workable. Do you have a dev site I could poke at to look at the implementation?

[lmorchard]

There are some other ways that might be more workable. Do you have a dev site I could poke at to look at the implementation?

Sure: https://mozilla.github.io/Themer/

[lmorchard]

FWIW, I think the main spots where we run into trouble with inline styles are these components:

https://github.com/mozilla/Themer/blob/master/src/web/lib/components/AppBackground/index.js#L16 https://github.com/mozilla/Themer/blob/master/src/web/lib/components/BrowserPreview/index.js

These want to apply dynamic styles for colors & background images based on user selections while building a theme.

[april]

In general, I don't have much issue with using 'unsafe-inline' inside style-src. It's not overly risky, and I understand that it can place a high complexity burden on sites that wish to avoid it.

There are a couple of anti-patterns (that we know of) that can allow attackers to do attacks purely in CSS, such as you can see here. Namely:

• Don't use JavaScript frameworks like React and set value directly on form elements, if the user can in any way inject inline <style> tags to the page.
• There are some weird interactions with fonts, iframes, and scrollbars, so be careful about where you allow iframes to be set and what you allow users to do in them.

Otherwise, the main risk with inline styles is phishing attacks. As long as you're very careful with how user inputs get translated onto the page, that can be largely avoided.

Note that the CSP3 working group is working on changes to CSP3 that would allow style= attributes while disallowing <style> tags, fixing most of the risk:

https://github.com/w3c/webappsec-csp/issues/45

But obviously that's a ways away, both from implementation in a browser and in the spec.

Based on what I see with themer, it seems like you should be able to do what you're doing now, without using inline styles. Here is a quick and dirty example:

https://www.twoevils.org/html/test/csp-set-style-no-unsafe-inline-style.html

Inline styles are blocked, but JavaScript is still updating the background color. Again, I'm not sure how complex it would be to rearchitect things to work with React in this manner. If that complexity is high, I'm personally not overly worried about 'unsafe-inline' remaining inside style-src.

[psiinon]

I'd recommend having a csp reporter even if theres no auth. Its a good way to find out if the site is broken, eg by a change thats got through testing but is still blocked by the CSP. Obviously this only helps if we monitor the CSP reports. What do we do for other services? Can we just do the same, but maybe with lower priority alerting?

[jbuck]

Right now I'm using a Lambda function to add custom security headers. I think it would be trivial to do something like have it forward CSP reports to Sentry which also has the advantage of notifying developers that something broke

[psiinon]

:+1:

[jvehent]

@jbuck FYI sending violation reports to a different location means that the blocked_uri field of the report will only contain the domain and not the full path. This is why we recommend sending violation reports to the same origin. I suppose you could proxy the reports to sentry and get the best of both worlds.

[jbuck]

Update: I was able to get CSP reports over to Sentry pretty easily. The problem is that Sentry currently doesn't accept CSP reports from Firefox because it's missing some fields in the payload.

[ghost]

@jbuck is that the right bug? I don't see anything about missing fields there

[jbuck]

bah, you're right, this is the correct bug: https://github.com/getsentry/sentry/issues/3542

[ghost]

@jbuck what's the next step? We can't report CSP errors from Firefox? Or we can just fake those fields in your Lambda function?

[jbuck]

@wresuolc hmm, we could definitely have a lambda function fake those fields, yeah. That seems like a reasonable approach here. I'll update this bug when that's done

[lmorchard]

Switched assignment, because I'm not actively working on this. I think #207 covers the last bit of work here.

[jbuck]

I have a patch up for review at https://github.com/mozilla-services/cloudops-deployment/pull/2013 that outputs any CSP reports to Cloudwatch Logs, which is a good enough starting point for the security review to go ahead. Once it's been reviewed, I'll deploy to production

[jbuck]

The CSP reporter has been running in production since launch, and I got a review so it's merged to master as well.