Security Checklist

[jvehent]

Risk Management

• [x] The service must have performed a Rapid Risk Assessment and have a Risk Record bug
• [x] "New Service issue" link is a 404 not found The service must be registered via a New Service issue

Infrastructure

• [ ] Access and application logs must be archived for a minimum of 90 days
• [x] Use Modern or Intermediate TLS
• [x] Set HSTS to 31536000 (1 year)
  • strict-transport-security: max-age=31536000
  • [x] If the service is not hosted under services.mozilla.com, it must be manually added to Firefox's preloaded pins.
• Admin panels will essentially be AWS admin - do we need Moz VPN & Auth0? If service has an admin panels, it must:
  • [x] only be available behind Mozilla VPN (which provides MFA)
  • [x] require Auth0 authentication

Development

• [x] Ensure your code repository is configured and located appropriately:
  • Only designated people should be allowed to push to production branches. (GitHub protected branches.)
  • Branch protections should always apply to administrators as well.
  • Host your repository in a trusted organization (one that follows EIS Recommendations). A list is maintained here.
  • Ensure all contributors are in compliance with the user guidelines
  • Elevated permissions should be granted to teams, not individual accounts, whenever possible. (Only org members can be part of a team.)
• [x] Sign all release tags, and ideally commits as well
  • Developers should configure git to sign all tags and upload their PGP fingerprint to https://login.mozilla.com
  • The signature verification will eventually become a requirement to shipping a release to staging & prod: the tag being deployed in the pipeline must have a matching tag in git signed by a project owner. This control is designed to reduce the risk of a 3rd party GitHub integration from compromising our source code.
• [x] enable security scanning of 3rd-party libraries and dependencies
  • Use nsp check for node.js (see usage in FxA and screenshots)
  • For Python, enable pyup security updates:
  • Add a pyup config to your repo (example config: https://github.com/mozilla-services/antenna/blob/master/.pyup.yml)
  • Enable branch protection for master and other development branches. Make sure the approved-mozilla-pyup-configuration team CANNOT push to those branches.
  • From the "add a team" dropdown for your repo /settings page
    • Add the "Approved Mozilla PyUp Configuration" team for your github org (e.g. for mozilla and mozilla-services)
    • Grant it write permission so it can make pull requests
  • notify secops@mozilla.com to enable the integration in pyup
• [x] Keep 3rd-party libraries up to date (in addition to the security updates)
  • For NodeJS applications, use renovate or [GreenKeeper](https://greenkeeper.io/ Greenkeeper)
  • For Python, use pip list --outdated or requires.io or pyup outdated checks
  • For Rust, use cargo update and cargo upgrade when changing versions
• [x] Integrate static code analysis in CI, and avoid merging code with issues
  • Javascript applications should use ESLint with the Mozilla ruleset
  • Python applications should use Bandit
  • Go applications should use the Go Meta Linter
  • Use whitelisting mechanisms in these tools to deal with false positives

Dual Sign Off

• [x] Services that push data to Firefox clients must require a dual sign off on every change, implemented in their admin panels
  • This mechanism must be reviewed and approved by the Firefox Operations Security team before being enabled in production

Logging

• [x] Amazon CloudWatch logging is used, do we need mozlog? Publish detailed logs in mozlog format (APP-MOZLOG)
  • Business logic must be logged with app specific codes (see FxA)
  • Access control failures must be logged at WARN level

Security Headers

• [x] There are no HTML pages used in this application Must have a CSP with
  • [x] a report-uri pointing to the service's own /__cspreport__ endpoint
  • [x] web API responses should return default-src 'none'; frame-ancestors 'none'; base-uri 'none'; report-uri /__cspreport__ to disallowing all content rendering, framing, and report violations
  • [x] if default-src is not none, frame-src, and object-src should be none or only allow specific origins
  • [x] no use of unsafe-inline or unsafe-eval in script-src, style-src, and img-src
• [x] Web APIs must set a non-HTML content-type on all responses, including 300s, 400s and 500s
• [x] No cookies Set the Secure and HTTPOnly flags on Cookies, and use sensible Expiration
• [x] We don't get an A, but this service should exist behind a firewall Make sure your application gets an A+ on the Mozilla Observatory
• [x] "Security Baseline" link is a 404 Not Found on github Verify your application doesn't have any failures on the Security Baseline.
  • Contact secops@ or ping 'psiinon' on github to document exceptions to the baseline, mark csrf exempt forms, etc.
• [x] Do we need an OpenAPI resource here? There is only one private web API endpoint, for internal Mozilla use Web APIs should export an OpenAPI (Swagger) to facilitate automated vulnerability tests

Security Features

• [x] Hawk auth is used for server-to-server requests, Amazon IAM for admin. Authentication of end-users should be via FxA. Authentication of Mozillians should be via Auth0/SSO. Any exceptions must be approved by the security team.
• [x] There are no sessions Session Management should be via existing and well regarded frameworks. In all cases you should contact the security team for a design and implementation review
  • Store session keys server side (typically in a db) so that they can be revoked immediately.
  • Session keys must be changed on login to prevent session fixation attacks.
  • Session cookies must have HttpOnly and Secure flags set and the SameSite attribute set to 'strict' or 'lax' (which allows external regular links to login).
  • For more information about potential pitfalls see the OWASP Session Management Cheat Sheet
• [x] Access control is just a flat allow / deny based on auth. Access Control should be via existing and well regarded frameworks. If you really do need to roll your own then contact the security team for a design and implementation review.
• [x] If you are building a core Firefox service, consider adding it to the list of restricted domains in the preference extensions.webextensions.restrictedDomains. This will prevent a malicious extension from being able to steal sensitive information from it, see bug 1415644.

Databases

• [x] SQL is not used All SQL queries must be parameterized, not concatenated
• [x] Applications must use accounts with limited GRANTS when connecting to databases
  • In particular, applications must not use admin or owner accounts, to decrease the impact of a sql injection vulnerability.

Common issues

• [x] No user data is presented in any context User data must be escaped for the right context prior to reflecting it
  • When inserting user generated html into an html context:
  • Python applications should use Bleach
  • Javascript applications should use DOMPurify
• [x] There are no user inputs Apply sensible limits to user inputs, see input validation
  • POST body size should be small (<500kB) unless explicitly needed
• [x] When managing permissions, make sure access controls are enforced server-side
• [x] If caching is used then make sure that any data cached does not incorrectly allow allow access to data protected by access control
• [x] There are no cryptographic keys used If handling cryptographic keys, must have a mechanism to handle quarterly key rotations
  • Keys used to sign sessions don't need a rotation mechanism if destroying all sessions is acceptable in case of emergency.
• [x] Do not proxy requests from users without strong limitations and filtering (see Pocket UserData vulnerability). Don't proxy requests to link local, loopback, or private networks or DNS that resolves to addresses in those ranges (i.e. 169.254.0.0/16, 127.0.0.0/8, 10.0.0.0/8, 100.64.0.0/10, 172.16.0.0/12, 192.168.0.0/16, 198.18.0.0/15).
• [x] Do not use target="_blank" in external links unless you also use rel="noopener noreferrer" (to prevent Reverse Tabnabbing)

cc @ameihm0912 to run the RRA

[lmorchard]

Also noting that nsp has changed hands and we probably want to switch to npm audit:

https://docs.npmjs.com/getting-started/running-a-security-audit

[lmorchard]

A kind of naive question on npm audit: It has found some issues in nested dependencies of dev dependencies. How should we handle these when the upstream projects haven't upgraded? Concerned that if we strictly fail the CI build on this, it effectively halts builds until other projects update.

➜  watchdog-proxy git:(3-process-queue) ✗ npm audit

                       === npm audit security report ===

┌──────────────────────────────────────────────────────────────────────────────┐
│                                Manual Review                                 │
│            Some vulnerabilities require your attention to resolve            │
│                                                                              │
│         Visit https://go.npm.me/audit-guide for additional guidance          │
└──────────────────────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ High          │ Denial of Service                                            │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ https-proxy-agent                                            │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=2.2.0                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ serverless [dev]                                             │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ serverless > https-proxy-agent                               │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://nodesecurity.io/advisories/593                       │
└───────────────┴──────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ Low           │ Prototype Pollution                                          │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ lodash                                                       │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=4.17.5                                                     │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ nsp [dev]                                                    │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ nsp > cli-table2 > lodash                                    │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://nodesecurity.io/advisories/577                       │
└───────────────┴──────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ Low           │ Prototype Pollution                                          │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ deep-extend                                                  │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=0.5.1                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ onchange [dev]                                               │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ onchange > chokidar > fsevents > node-pre-gyp > rc >         │
│               │ deep-extend                                                  │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://nodesecurity.io/advisories/612                       │
└───────────────┴──────────────────────────────────────────────────────────────┘

[!] 3 vulnerabilities found - Packages audited: 4583 (4480 dev, 111 optional)
    Severity: 2 Low | 1 High

[pdehaan]

Per https://blog.npmjs.org/post/173719309445/npm-audit-identify-and-fix-insecure

What if I’m using a previous version of npm?

npm audit is available in npm@5.10.0 and npm@6. Prior versions of npm will receive vulnerability messages similar to the following:

npm-notice: [SECURITY] marked has 3 high, and 2 moderate vulnerabilities. Go here for more details: https://nodesecurity.io/advisories?search=marked&version=0.3.0 - Run npm i npm@latest -g to upgrade your npm version, and then npm audit to get more info.

We’re not aware of any third-party registry clients that currently support displaying the npm-notify header, so users of these tools will not receive vulnerability messages. For maximum protection against unsafe code, all users should use npm@6.

Looks like the latest Node 8.11.1 comes w/ npm@5.6.0. Heck, even the latest node 10.1.0 comes w/ the older npm@5.6.0, so we'd have to manually install npm@5.10+ or npm@6.

Also, doesn't look like we can currently ask npm audit to ignore devDependencies (per npm/npm #20564).

Also also, doesn't look like you can currently have a list of exceptions (per npm/npm #20565).

[lmorchard]

Took another swing through the checklist - I think most things are covered off with the exception of things to handle during prod / stage deployment. A few remaining questions:

• Admin panels will essentially be AWS admin - do we need Moz VPN & Auth0?
• "New Service issue" link is a 404 not found - what needs to happen there?
• Amazon CloudWatch logging is used, do we need mozlog?
• "Security Baseline" link is a 404 Not Found on github - what things need to happen there?
• Do we need an OpenAPI resource here? There is only one private web API endpoint (POST /accept) for internal Mozilla use

For summary clarification: This project is a server-to-server API with no end-user web UI beyond AWS admin pages. A Mozilla consumer site POSTs submissions to a rate limiting queue, which POSTs to an external web service and then POSTs results back to the consumer site.

[ameihm0912]

Amazon CloudWatch logging is used, do we need mozlog?

Ideally we'd want mozlog here, this would save us from having to write a new decoder for the log data.

Since the log data is in CloudWatch we will likely want to look at hooking that feed up to a Kinesis stream for consumption as part of the stack deployment.

[ameihm0912]

The RRA for watchdog has been completed

[oremj]

SSL Report: https://www.ssllabs.com/ssltest/analyze.html?d=watchdog-proxy.services.mozilla.com&s=54.200.32.91&latest

[clouserw]

No admin panels, so I'm checking those boxes off

[clouserw]

Since the log data is in CloudWatch we will likely want to look at hooking that feed up to a Kinesis stream for consumption as part of the stack deployment.

@ameihm0912 I'm not sure the details of that. Is it instead of us converting the application to use mozlog or in addition to?

[lmorchard]

Just saw HSTS filed as a separate issue (#191) - but do we actually need HSTS here? This isn't a service or site intended for use in a browser, and HSTS looks like it's intended to tell browsers to upgrade from HTTP to HTTPS connections. This is a server-to-server microservice, so we should probably just deny http connections altogether (if we're not already)

[lmorchard]

Also there are still other open questions on this issue for remaining checkboxes. Not sure who to ping about them. /cc @jvehent? @ameihm0912?

[ameihm0912]

@clouserw @lmorchard the thought here was to get a feed of log data from the service we can analyze for abuse. Getting the feed is probably more important than the mozlog conversion; I can create a bug for this.

[clouserw]

I checked off HSTS here, in favor of #191

[clouserw]

I filed https://github.com/mozilla-services/foxsec/issues/1074 to check off the inform checkbox.

@oremj: Are you able to confirm:

• Access and application logs must be archived for a minimum of 90 days
• Use Modern or Intermediate TLS

@jvehent / @moz-jvehent

• We're using Amazon's CloudWatch. Do we need mozlog?
• Do we need to run the security baseline w/ @psiinon? This is an internal API
• Do we need an OpenAPI resource here? Again, this is an internal API

[jvehent]

We're using Amazon's CloudWatch. Do we need mozlog?

Yes. Mozlog is the format, cloudwatch is the transport/storage. Ultimately, we want those logs to be treated like standard application logs.

Do we need to run the security baseline w/ @psiinon? This is an internal API Do we need an OpenAPI resource here? Again, this is an internal API

No. You may if you want to, but those recommendations are primarily for public endpoints.

[clouserw]

I filed #196 to use the mozlog format and I'll check it off the list above

[clouserw]

@oremj . Are you able to confirm?

• Access and application logs must be archived for a minimum of 90 days
• Use Modern or Intermediate TLS

[oremj]

I will double check log retention.

I'm not up to date on Modern/Intermediate TLS definitions, but here is the current state: https://www.ssllabs.com/ssltest/analyze.html?d=watchdog-proxy.services.mozilla.com&hideResults=on&latest

[clouserw]

Thanks jeremy. @jvehent that ssllabs link seems to think we're doing alright. Would you confirm we're using modern TLS? ^

[jvehent]

You're on intermediate TLS, which is good enough.

• Mozilla evaluation: intermediate

[oremj]

The logs are currently going to cloudwatch logs and sticking around for longer than 90 days. We need to figure out a way to find the new log groups that spin up and set an expiration on them.

How critical is this?