It adds a new method in Scraper, update_form_input, to update/create the form_input.tsv. Primarily, this remove DataSpace records where they have been withdrawn, which is primarily causing #49. Second, it also adds new entries. By addressing both record updates, this removes the need to manually update the TSV file, particularly with new entries.
Second, with new records, it automatically updates the DOE metadata (sponsoring organizations, funding, and datatype) with default settings. This ensure that Poster will run successfully.
Note that this is toward having the ability for CI to scrape these data and update records, removing the need for manual involvements as much as possible.
Closes #49
This PR does two things:
Scraper,update_form_input, to update/create the form_input.tsv. Primarily, this remove DataSpace records where they have been withdrawn, which is primarily causing #49. Second, it also adds new entries. By addressing both record updates, this removes the need to manually update the TSV file, particularly with new entries.Posterwill run successfully.Note that this is toward having the ability for CI to scrape these data and update records, removing the need for manual involvements as much as possible.