Improve scraper for Office of Finance and Operations (Phase 2) - Githubissues

CivicActions / edscrapers

US Department of Education Data Scraping Kit; see https://us-ed-scraping.ckan.io/dataset

GNU Affero General Public License v3.0

15 stars 9 forks source link

Improve scraper for Office of Finance and Operations (Phase 2) #169

Open higorspinto opened 4 years ago

higorspinto commented 4 years ago

During phase 1 we created a functional scraper for crawling and parsing data from this office. The scraped data was successfully ingested into the data portal.

For phase 2, we need to improve the quality of metadata and data-content for the datasets being generated by the scraper.

https://www2.ed.gov/about/offices/list/ofo/index.html

Acceptance Criteria

[ ] we have marked improvement in the quality of metadata and data-content of datasets produced by the scraper.
[ ] the improved quality datasets are visible on the data portal

Tasks

[ ] Ensure datasets produced have a description metadata
[ ] Ensure datasets have a publisher metadata
[ ] Improve other metadata (use defaults where available)