search

GSA / data.gov

Main repository for the data.gov service
https://data.gov
Other
645 stars 101 forks source link

Research 3d: Identify low-lift effort for a new evidence act compliance information #1757

Closed adborden closed 4 years ago

adborden commented 4 years ago

User Story

In order to remove the dashboard and focus efforts on high-value delivery of the Data.gov mission, Data.gov wants to create a maintainable, modern source of information (similar to the dashboard) for OMB to report to GAO.

In order to provide essential information for Evidence Act compliance, OMB wants a single public location of information for reporting to GAO.

Tasks

Background

The dashboard consumes more effort from the Data.gov team than the value it is delivering to users. The audience for the Dashboard is unclear, we know OMB uses some of the dashboard functionality to prepare for GAO meetings.

June 16th, 2020 conversation with OMB.

By focusing the information that Dashboard collects to the essentials for OMB, we can create a new solution with minimal effort. Hopefully this means:

Security Considerations (required)

None for this research story. Any stories created from this work should identify any security considerations.

Sketch

Catalog already provides some "meta" datasets about the catalog (e.g. bulk metadata download, site statistics) which are implemented as a CKAN extension and run regularly with cron. We could treat the Evidence Act compliance information in a similar way, a dataset available through catalog.data.gov (or inventory.data.gov). IMHO, it's an accident of history that dashboard crawl data is hidden under this "labs" feature with little documentation and visibility instead of being available in the catalog.

Perhaps a data file containing this compliance information is enough for OMBs needs. A static HTML/JS dashboard could pull this data file for visual rending. If the data file is sufficient, the dashboard could be implemented at a later date and would still be able to provide historic data would be available once we start collecting data for the dataset.

jbrown-xentity commented 4 years ago

Per Aaron and OMB conversation, the key items we need are:

Items that are currently on the Dashboard:

jbrown-xentity commented 4 years ago

First pass at documenting current state, reason for desired changes, and other possible implementations that would require less management can be found here.

jbrown-xentity commented 4 years ago

Next steps include building out a quick test script validating CKAN API usage for building a CSV, and possibly building out a dynamic representation of data from an inventory API as a static page (possibly resources.data.gov, in staging only?)

pjsharpe07 commented 4 years ago

Current version of a simple crawl that produces a csv can be found here along with some notes.

adborden commented 4 years ago

@pjsharpe07 can you include an example CSV output we can share with OMB?

mogul commented 4 years ago

(CSV is in the Google Drive link above.)

mogul commented 4 years ago

Unblocking until our sketch of how to proceed from here is complete.

mogul commented 4 years ago

Planning to discuss with Rebecca ahead of next Friday.

mogul commented 4 years ago

There appear to be no strong obstacles or objections to our proposed approach. We'll check with Phil as we go, but generally the research story can be concluded.

nickumia-reisys commented 1 year ago

Related to