search

cloud-gov / product

Program-level artifacts, workflow and issues for cloud.gov
Creative Commons Zero v1.0 Universal
31 stars 15 forks source link

Run October 2023 ConMon scans #2750

Closed Chiakao closed 6 months ago

Chiakao commented 1 year ago

In order for us to update the JAB on our compliance in a consistent way, we need to run Continuous Monitoring scans on approximately the 23rd of the month. (If this date falls on a weekend or federal holiday, adjust to the last business day before the date.)

For context, see our Continuous Monitoring Strategy, including the monthly reporting summary explanation.

Netsparker

Omit: I am not finding a clear path to using Netsparker for our admin apps, and NetSparker has its own issues with false positives

OWASP ZAP Scans

From the ZAP documentation: "Zed Attack Proxy (ZAP) is a free, open-source penetration testing tool being maintained under the umbrella of the Open Web Application Security Project (OWASP). ZAP is designed specifically for testing web applications and is both flexible and extensible."

We run ZAP from a platform operator's local machine. ZAP opens a Firefox instance that is configured to proxy all requests through ZAP. ZAP can then analyze and modify the requests.

Preliminary work - Sandbox User Setup

We scan our externally facing apps as sandbox users of cloud.gov, via the cloud.gov IdP, instead of platform admins. This vastly speeds up scans since the spider doesn't crawl every app and org in logs or dashboard, and also avoids issues with ZAP "clicking" on links with undesired impacts.

Create a sandbox account, starting from https://account.fr.cloud.gov/signup, and use an email such as your fname.lname@{cio.gov, pif.gov, fedramp.gov}

As your "sandbox" user identity, launch a "Hello World" app so there's something in the dashboard and logs apps to spider. (ToDo: Determine if this is really necessary, provide link to steps).

Install, Configure, and Update

Quit and restart ZAP if you change the JVM options.

Running ZAP scans

ZAP scans take hours. We recommend you start in the morning or run them overnight. There are two separate scans to run, external and internal, and the internal one takes considerably longer. (You may want to run it when VPN traffic is lower.)

The following steps are for the external scan (except as noted):

Quit ZAP, then repeat the "Running ZAP scans" steps for the internal context (which will require the VPN)

Troubleshooting ZAP Scans

502 Bad Gateway, ZAP Error

If you encounter the following:

ZAP Error [org.apache.hc.core5.http.NoHttpResponseException]

Make sure you are typing the entire URL, including the https:// protocol, into Firefox. Firefox will appear to automatically redirect from http to https, but if you check the ZAP console, you'll see the requests being made in http and failing with 502 Bad Gateway.

Browser Was Not Found, Java Exceptions

Java Unable to Connect Exception

In Firefox if you see a Java Unable to Connect Exception, try the following:

Close both Firefox and ZAP.

In ~/Library/Application Support/ZAP/log4j2.properties:

Change the following level's to debug so the entries look like this:

logger.paros.name = org.parosproxy.paros
logger.paros.level = debug

logger.zap.name = org.zaproxy.zap
logger.zap.level = debug

Open ZAP, follow the above and open Firefox. Try to go to the server that failed previously.

If that works, then change the levels back to info from debug, so they look like this:

logger.paros.name = org.parosproxy.paros
logger.paros.level = info

logger.zap.name = org.zaproxy.zap
logger.zap.level = info

For the internal sites, try the following order in Firefox to bring up the sites according to the context:

https://ci.fr.cloud.gov
https://alertmanager.fr.cloud.gov
https://logs-platform.fr.cloud.gov
https://grafana.fr.cloud.gov
https://prometheus.fr.cloud.gov
https://opslogin.fr.cloud.gov

If the context changes the sites, this list and order will need to be revisited.

Generic Troubleshooting

For when the other troubleshooting has not helped:

Upload and wrap up

Upload all reports to Google Drive: https://drive.google.com/drive/u/0/folders/0B5fn0WMJaYDnaFdCak5WNWRGb1U in a folder named YYYYMMDD-ZAP-Nessus.

You can shut down ZAP and Firefox.

Potential ZAP Issues

Acceptance criteria

Disk Usage

A single ZAP scan of the cloud-gov context requires significant disk space (over 100GB). If you have run ZAP previously, you should check to see if you previous sessions have been persisted. If so, you likely need to clear out those directories before proceeding.

You can check ZAP's disk usage with:

du -h -d 1 ~/Library/Application\ Support/ZAP/

If you see an abnormally large session or sessions directory (my last run was 132G), you likely want to delete all files in these directories before proceeding. Choosing to "Not persist" sessions should alleviate this issue.

Export Nessus Scans

Acceptance criteria:

The following (.nessus and .html) are all uploaded to YYYYMMDD-ZAP-Nessus folder:

Update the POAM Inventory sheet

A python script is used to generate the inventory list.

wz-gsa commented 6 months ago

Work complete