Adds the pipeline to strip out the unnecessary variables from the annotated patents/publications.
Includes test for added functions.
I purposefully haven't included getters as I'm not sure these files are useful for analysis (but the post processed files will)
Instructions for Reviewer
Run python dap_aria_mapping/pipeline/entity_processing/reformat_entities.py --datastore=s3 run to run the flow in test mode
Run pytest on the test file
Checklist:
[x] I have refactored my code out from notebooks/
[x] I have checked the code runs
[x] I have tested the code
[x] I have run pre-commit and addressed any issues not automatically fixed
[x] I have merged any new changes from dev
[x] I have documented the code
[x] Major functions have docstrings
[x] Appropriate information has been added to READMEs
[x] I have explained this PR above
[x] I have requested a code review
THE ONLY CHANGED FILES ARE dap_aria_mapping/pipeline/entity_processing/reformat_entities.py AND dap_aria_mapping/pipeline/entity_processing/tests/test_reformat_entities.py
Description
Adds the pipeline to strip out the unnecessary variables from the annotated patents/publications. Includes test for added functions. I purposefully haven't included getters as I'm not sure these files are useful for analysis (but the post processed files will)
Instructions for Reviewer
Run
python dap_aria_mapping/pipeline/entity_processing/reformat_entities.py --datastore=s3 runto run the flow in test mode Run pytest on the test fileChecklist:
notebooks/pre-commitand addressed any issues not automatically fixeddevREADMEsTHE ONLY CHANGED FILES ARE
dap_aria_mapping/pipeline/entity_processing/reformat_entities.pyANDdap_aria_mapping/pipeline/entity_processing/tests/test_reformat_entities.py