idn-au / catalogue-data

Creative Commons Attribution 4.0 International
0 stars 1 forks source link

Automated pipelines for validation & updating data #8

Closed jamiefeiss closed 1 year ago

jamiefeiss commented 1 year ago

Implemented the following workflows:

Both workflows check which files were modified so only the appropriate steps are run when necessary, i.e. if a vocab was modified then the catalogs are not validated & re-uploaded, or if only the README was modified then nothing should run.

The update script uploads the data for each file into a separate named graph for each vocab, catalog & resource, and all background data is placed into a single named graph.

I will also be setting some branch rules to protect ourselves with these workflows in place.

jamiefeiss commented 1 year ago

I've set the branch rules now. The PR check has failed because of SHACL validation on the catalogue data. Is the validator up-to-date? In the validation script I've grabbed the validator from https://raw.githubusercontent.com/idn-au/idn-catalogue-profile/main/rd/validator.ttl