Add more data links to /data

[gbinal]

There's some more options here.

[gbinal]

Note that there are other, unpublished data links that would be good to include, a la:

/static/data/tables/https/https-domains.csv /data/domains/analytics.json

More can be found by searching the repo for e.g. GET commands in the scripts.

[gbinal]

Here are some more -

• https://pulse.cio.gov//data/agencies/https.json

• https://pulse.cio.gov//data/domains/https.json

• https://pulse.cio.gov//data/domains/analytics.json

• https://s3-us-gov-west-1.amazonaws.com/cg-4adefb86-dadb-4ecf-be3e-f1c7b4f6d084/live/scan/analytics.csv

• https://s3-us-gov-west-1.amazonaws.com/cg-4adefb86-dadb-4ecf-be3e-f1c7b4f6d084/live/scan/pshtt.csv

• https://s3-us-gov-west-1.amazonaws.com/cg-4adefb86-dadb-4ecf-be3e-f1c7b4f6d084/live/scan/tls.csv

• https://s3-us-gov-west-1.amazonaws.com/cg-4adefb86-dadb-4ecf-be3e-f1c7b4f6d084/live/scan/sslyze.csv

[gbinal]

This will be great to do but is largely predicated on getting a 👍 to publish the other sections. I'm adding to the backlog but otherwise going to go ahead and close it for now.

[gbinal]

Using this issue to try and reference all of the hosted data files that I'm aware of.

Live

• Info for https domain section of pulse (~1k entries) - https://pulse.cio.gov/data/domains/https.json
• Info for https agencies section of pulse (~130 entries) - https://pulse.cio.gov/data/agencies/https.json
• HTTPS scan data downloads (~1k entries) - https://pulse.cio.gov/data/domains/https.csv
• Domain Scan data downloads (~1300 entries) - https://s3.amazonaws.com/pulse.cio.gov/live/scan/inspect.csv
• HTTPS scan data (~25k entries) - https://pulse.cio.gov/data/hosts/https.csv
• HTTPS scan data (~25k entries) - https://pulse.cio.gov/data/hosts/https.json
• HTTPS scan data (~1300 entries) - https://s3.amazonaws.com/pulse.cio.gov/live/scan/pshtt.csv
• HTTPS scan data 2 (~1600 entries) - https://s3.amazonaws.com/pulse.cio.gov/live/scan/tls.csv
• DAP data results (~600 entries - https://pulse.cio.gov/data/hosts/analytics.csv
• DAP data results (~600 entries - https://pulse.cio.gov/data/hosts/analytics.json
• DAP initial scan results (~1300 entries) - https://s3.amazonaws.com/pulse.cio.gov/live/scan/analytics.csv
• Pulse DB file (~1300 entries) - https://s3.amazonaws.com/pulse.cio.gov/live/db/db.json
• donut data - https - https://pulse.cio.gov/data/reports/https.json
• donut data - analytics - https://pulse.cio.gov/data/reports/analytics.json

Staging

• a11y individual error details - https://staging.pulse.cio.gov/static/data/tables/accessibility/a11y.json
• agencies and error totals - https://staging.pulse.cio.gov/static/data/tables/accessibility/agencies.json
• domains and error totals - https://staging.pulse.cio.gov/static/data/tables/accessibility/domains.json
• https://staging.pulse.cio.gov/static/data/tables/accessibility/domain-errors.csv
• https://staging.pulse.cio.gov/static/data/tables/accessibility/https-domains.csv
• https://staging.pulse.cio.gov/static/data/tables/accessibility/inspect.csv
• https://staging.pulse.cio.gov/static/data/tables/accessibility/pshtt.csv

Suspected but no longer work:

• Resolves but empty - https://pulse.cio.gov//data/domains/analytics.json
• Resolves but empty - https://pulse.cio.gov/data/domains/analytics.csv
• Access denied (likely dup of above) - https://s3-us-gov-west-1.amazonaws.com/cg-4adefb86-dadb-4ecf-be3e-f1c7b4f6d084/live/scan/inspect.csv
• Exists but times out - https://staging.pulse.cio.gov/static/data/tables/accessibility/a11y.csv