[feat]: CPO-206

[vpchung]

Changelog

• Use CSV file generated by data-models for most up-to-date list of standard terms (rather than Synapse table which is no longer maintained)
• Add logic to remove duplicates (if found) after mapping legacy annotations to the latest standard terms
• Add dryrun parameter, where if enabled, CSVs of the updated tables are created in lieu of updating the Synapse tables
• General code maintenance, e.g. add type hinting, update functions, update ATTRIBUTE_DICT in attribution_dictionary, etc.

Preview

I used the following dummy tables for testing the script:

• Manifest 1
• Manifest 2
• Manifest_UNION

Before update

Expected updates:

• All values should be considered, even if they're in a list. For example:
  • "Modeling" is the current standard term, no re-mapping needed
  • "Medip" should be updated to "MeDIP"
• "Mint-chip" should be updated to "Mint-ChIP"
• "Malignant Glial Tumor" should be updated to "Malignant Glioma"
  • Special case: after re-mapping, "Malignant Glioma" will appear twice for the first row, therefore the duplicate annotation should also be removed
• "Cancer of Bladder" should be updated to "Bladder Carcinoma"

After update

All expected updates have been applied.

If --dryrun is enabled

$ python annotations/edit_legacy_annotations.py -u syn61486342 --dryrun   

❗❗❗ WARNING: dryrun is enabled. Results will be saved to CSV instead.
================================================================================

Updating annotations found in table ID: syn61486245
        Checking assay...
        Checking tumorType...
Updating annotations found in table ID: syn61486250
        Checking assay...
        Checking tumorType...

where a sample CSV will look something like this:

name,Publication Assay,Publication Tumor Type,entityId
Verena,"MeDIP, Modeling",Malignant Glioma,syn61486245
Test,Forster Resonance Energy Transfer,Bladder Carcinoma,syn61486245

[vpchung]

@Bankso @aditigopalan OK, I did a couple more test runs and I think it is working as expected.

If y'all can think of any other edge cases we should account for, just lmk! I will merge this in the meantime.

Tagging @aclayton555 for awareness as well.