Open mikehgrantsgov opened 5 days ago
A few small suggestions:
extract_metadata_id
instead of id
for the primary key - also make it a BigInteger to follow the pattern of other tables (doubt we hit the ~4 billion limit of a regular int, but just in case)file_size_bytes
as BigInteger).opportunities_xml
and opportunities_csv
as values?)Thanks, adapted this!
Related to 2454
Summary
The legacy website provides the ability to download XML extracts of opportunity data: https://www.grants.gov/xml-extract - this is something we should also support. Having extracts (especially if they're just the same data as our search endpoints) reduces the traffic our search endpoint will need to handle as we can point anyone that would want to scrape our data to an extract file.
Create a table to track extract metadata.
Something like:
Acceptance criteria