search

GSA / data

Assorted data from the General Services Administration.
2.11k stars 275 forks source link

Rapid7 Open Data #190

Closed hrbrmstr closed 4 years ago

hrbrmstr commented 5 years ago

Greetings!

I'm Rapid7 Labs' Chief Data Scientist and you can get free FDNS data via https://opendata.rapid7.com/ (i.e. you can more regularly update the Rapid7-derived FDNS data set).

Just go to the site, hit one of the data boxes then use the signup form. The Labs team generally responds within 48hrs.

(hopefully the folks that monitor this GH aren't in the furloughed category)

-hrbrmstr

IanLee1521 commented 5 years ago

Hi @hrbrmstr -- I'm not an admin on this repo, but I had pointed the efforts to the Rapid7 data in the past. Some of that is pulled in to https://github.com/GSA/data/tree/master/dotgov-websites (see bullet 2).

I like the idea of automating the pulling of the data from Rapid7, but there are some issues given the volume and needing such a small (.gov / .mil) slice of the domains.

hrbrmstr commented 5 years ago

It would be very straightforward for us to write a job that does a .gov / .mil export every time (~2wkd) the FDNS study runs and submit a PR to here with it if that sounds like something you'd be interested in.

IanLee1521 commented 5 years ago

I'm not the owner of the repo, but I think something like a pull request to update https://github.com/GSA/data/blob/master/dotgov-websites/rdns-federal-snapshot.csv would be valuable, yes.

h-m-f-t commented 5 years ago

Hi @hrbrmstr. Super late response, but this post occurred during a government shutdown, and... somehow it is 6 months later.

Over at @cisagov, we'd be really interested in a pull of .gov hostnames seen in your scans if that's a thing you'd be interested in managing. Happy to chat here or at cameron.dixon@trio.dhs.gov.

jsf9k commented 5 years ago

@hrbrmstr I'm a colleague of @h-m-f-t's. We would definitely use this data if it were submitted here, or to a repo in the @cisagov organization.

Submitting the data to a repo in the @cisagov organization would be better for us, since none of us have the authority to approve PRs and perform merges in GSA/data. Thoughts, @h-m-f-t?