Reusable workflow fetch-and-push-to-artsdata.yml

[saumier]

I would like to have a reusable workflow that includes all the user inputs including headless-browser.

Please update the fetch-and-push-to-artsdata.yml and add what is missing, then refactor all the workflows to use it.

[dev-aravind]

@saumier All the workflows in Orion are now using a reusable workflow to fetch the data, including the headless one. Please test it and let me know if you find any issues.

[saumier]

Looking really good. A few things I noticed:

• [x] rcaaq-events.yml seems to have extra code - remove fetch-and-commit-data?
• [x] indent for on: has extra tab - remove one tab
• [x] rename fetch-and-push-to-artsdata.yml to fetch-data.yml because it only fetches data (I like this because it enables flexibility to add intermediate steps)

Also, can you make an additional workflow that combines the fetch and push parts so when there are no intermediate steps (like in replace-location-name) then it can all be a single workflow?

• [x] additional workflow fetch-and-push-to-artsdata.yml (same name but with additional included push)

[dev-aravind]

@saumier The workflow is now added. Let me know if you want to update all the crawl workflow to use this.

[saumier]

@dev-aravind I am thinking that the fetch-and-push-to-artsdata.yml should reuse the fetch-data.yml so it does not repeat. We will be keeping both the fetch-data.yml and the fetch-and-push-to-artsdasta.yml so in cases like capitol-nb-ca you can still choose to add custom steps. If you agree then please fix fetch-and-push-to-artsdata.yml.

[dev-aravind]

@saumier All the workflows now use the reusable workflow to fetch and push the data.

[saumier]

Looks good.