ONSdigital / dp-data-pipelines

Pipeline specific python scripts and tooling for automated website data ingress.
MIT License
1 stars 0 forks source link

Jim/validate transform #91

Closed JimBryant7 closed 4 months ago

JimBryant7 commented 5 months ago

What

Added validation and unit tests for the sdmx transform. Also implemented the validation into the transform. The validation functions are mainly checking that 'things' are as the expect to be ie I'm expecting a list of dicts and each dict should have these keys.

How to review

Check the unit tests, check that the transform runs without issues - the transform should work on data that has been tried before. I welcome any other validation steps that you think I have missed but this isn't a requirement :)

Who can review

It may make more sense to a DE but anyone can review.