ONSdigital / dp-data-pipelines

Pipeline specific python scripts and tooling for automated website data ingress.
MIT License
1 stars 0 forks source link

#84 initial acceptance test #95

Closed NickPapONS closed 4 months ago

NickPapONS commented 5 months ago

What

This implements more of the acceptance test steps based on the groundwork done in the acceptance tests fixture ticket: https://github.com/ONSdigital/dp-data-pipelines/pull/71

It adds steps that test the outputs generated by v1_data_ingress, mainly the csv and metadata.json output files.

Another significant change in this is the new use of a temporary directory in the behaviour tests. This is created at the start of a scenario, and is used to accommodate the outputting of files during tests, to allow them to be placed in a separate directory which can be safely deleted after each scenario is done. (The output files are often placed in the "current directory", which previously was simply the root directory and so it would be difficult to delete the files there.)

How to review

Try the behaviour tests and see if they pass in your environment. Look through the steps and see if we missed something or if something could be done better.

Who can review

Anyone other than Shannon and Nick (both worked on this)