ONSdigital / dp-data-pipelines

Pipeline specific python scripts and tooling for automated website data ingress.
MIT License
1 stars 0 forks source link

Refactor transform repetitive functions (verified) #166

Closed NickPapONS closed 3 weeks ago

NickPapONS commented 3 weeks ago

Recreated pull request with changes from https://github.com/ONSdigital/dp-data-pipelines/pull/160 due to unverified merge commit.

What

Refactored the three functions in transform.py into one function. The three functions used to to the same thing but on different target fields of the input config dictionary. The new function takes in an argument for what target field of the dictionary to work on, cutting down on the repetitive code in transform.py and also allowing us to remove some repetitive unit tests that were associated with the three old functions. How to review

Look through the changed transform.py code and see if it makes sense/if you agree with the approach I took. Run the tests and see if it all works. Who can review

Anyone