GSA / data.gov

Main repository for the data.gov service
https://data.gov
Other
604 stars 96 forks source link

[4d] Investigate Harvest Source CMS Options #4618

Closed btylerburton closed 6 months ago

btylerburton commented 7 months ago

Purpose

We want to administer harvest source configs, but we're not sure if using a CMS is better than just writing our own SQLALchemy interface on top of the Harvest DB

Given the above, conducting research/investigation/prototyping is needed to provide factual knowledge on future steps.

4d of effort has been allocated and once compete, findings will be demonstrated and specific future actions will be decided.

Acceptance Criteria

[ACs should be clearly demo-able/verifiable whenever possible. Try specifying them using BDD.]

GIVEN I have a created a model for harvest source configs AND it is represented in a CMS WHEN I add a new source THEN I should be able to query an API using that source's uniqueId and extract the relevant harvest source information

GIVEN I have the above API, WHEN I update the harvest source THEN I want a callback event to fire, which triggers a programmatic extraction of the relevant info AND insertion of that data into the Harvest DB

Background

NOTE that the CMS DB and Harvest DB should remain independent of each other as the schema a CMS uses will contain metadata that is not relevant to the Harvest DB table

Sketch

btylerburton commented 6 months ago

We've decided to use Sveltekit for UI and Flask for our admin frontend. Closing.