Task: View New Airflow Produced Output in Drafter

GSS-Cogs / DataEngineering_Airflow_Alpha

0 stars 0 forks source link

Task: View New Airflow Produced Output in Drafter #20

Open MartynBSpooner opened 1 year ago

MartynBSpooner commented 1 year ago

Tasks:

[x] Implement minimal pipeline steps (eg csvcubed, csvlint, csv2rdf)
[x] Add connection to PMD drafter using API keys
[x] Implement first pass of drafter upload queries
[ ] Identify any issues with dataset as it appears in the drafter
[ ] Implement any changes needed to have draft appear correctly in drafter (with existing dataset as reference)
[ ] Something something validation for various dataset structures (TBD)

MartynBSpooner commented 1 year ago

Waiting on Swirrl / TPX Impact for a response to why there are possible errors with our JSON, which on check appears to be valid by both J and Ross, but does not permit viewable drafts in drafter component.

MartynBSpooner commented 1 year ago

Task breakdown to be added - note that majority of code transfer completed against the Epic by J.