Complete EPA downloads - Githubissues

edgi-govdata-archiving / overview

🎈 Start here for current projects, how to get involved, and joining community calls, a resource for new and veteran members

GNU General Public License v3.0

118 stars 20 forks source link

Complete EPA downloads #119

Closed titaniumbones closed 5 years ago

titaniumbones commented 7 years ago

With the EPA in crisis, we need to finish off our efforts to gather EPA datasets together.

Minimum success criterion: archive copies of all known EPA datasets currently available on the open web.

Stretch goal: recreate opendata.epa.gov and potentially other developer & user interfaces as well.

Resources

Direct Contact

Despite many discussions about this, it does not appear that EDGI or Archivers has yet made a systematic effort to reach out to EPA sysadmins and see if we can grab datasets that way. We have probably reached the time to start doing this.

Tagging @b5, @dcwalk , @patcon to continue this conversation as we go forward.

dcwalk commented 7 years ago

Current status as of 2017-08-07:

~300 in archivers.space as URLs to research
crawling from datatogether informing https://github.com/datatogether/coverage
see #120 for discussion of how coverage is being calculated/generated

dcwalk commented 7 years ago

Reviewed this issue at Aug 14 archiving call:

By Aug 29, ideal world we have a Data Together Node that resolves this 2 pieces:

Coverage Analysis (static)

Storage of Data (Doris Duke deliverables [mostly not EPA], Crawling EPA)

Note: This is related to Doris Duke deliverables (need an issue to track that)

TODO:

[ ] Doris Duke in Datatogether @b5 (ticketed to #194)
[ ] Refresh our machine-readable list of who has what (coverage repos) opportunity for peer work? (@b5 @patcon) (ticketed to #196)
[ ] Review intersection of archives.space and datatogether.org ?? (wait on one above) (ticketed to #197
[ ] (Fall) Re-reach out to look at Project Azimuth, Climate Mirror, IA (ticketed to #195)

Open threads:

Quality? QA
What do we mean when we say coverage

dcwalk commented 7 years ago

Once #194, #196, #197, and #199 (maybe #195) are closed this can be resolved.

In the interim, moving to the Fall Work Cycle milestone based on our September 11 Archiving call...as this was indicated as an ongoing and important priority.

Frijol commented 5 years ago

Per our new stale issues policy:

This issue has been marked as stale because it has not had recent activity. It will be closed in seven days if no further activity occurs. If it should not be closed, please comment! Thank you for your contributions.

In the future, a robot will take care of this process!

Frijol commented 5 years ago

Per @dcwalk 's comment above, all the issues mentioned are closed except https://github.com/edgi-govdata-archiving/overview/issues/197 and https://github.com/edgi-govdata-archiving/overview/issues/195 (which is stale).

stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in seven days if no further activity occurs. If it should not be closed, please comment! Thank you for your contributions.