Sage-Bionetworks / schematic

Package for biomedical data model and metadata ingress management
https://schematicpy.readthedocs.io/en/stable/cli_reference.html
MIT License
22 stars 26 forks source link

RDB: Camelcase mismatch between Schematic and RDB uses. #785

Closed mialy-defelice closed 1 week ago

mialy-defelice commented 2 years ago

Describe the bug There is a mismatch in camelcase between how the RDB creates the names and how Schematic creates the names when generating the JSONLD.

Currently there is a workaround that I added to sql.py:

        # Hard coding a solution for right now to get the NF Table loaded.
        if 'howToAcquire' in input_table.columns:
            input_table.rename(columns={'howToAcquire': 'howtoAcquire'}, inplace=True)

That allows us to update the database with the Manifest data. Without it, there would be no error but the data would not load.

The dictionary we use in the script sql_query_nf.py are used to update the camelcase from the Database to any specific naming that might be required for proper loading in a Portal. See #757 for details.

To Reproduce Download all NF Tools Registry Manifests and their new data model that both contain the column How to Acquire. After making the NF Tools Registry database, try to load all the manifests.

The JSONLD will have returned the name howtoAcquire (this will then become the name of the column in the database). while the manifest will load with the column name howToAcquire. If you log into your database and look for the column howtoAcquire you will notice it is missing all the data. The workaround changes the manifest naming to match the database naming. Then we use the dictionary COL_NAME_UPDATES in sql_query_nf.py to convert this name back since that is what was needed for the portal.

Expected behavior Camelcase names are all made consistently.

I suspect this issue is due to using get_class_label_from_display_name when generating the JSONLD for this particular attribute while the RDB only uses get_property_label_from_display_name. But I have not tested this yet.

Priority (select one)

@andrewelamb tagging you for visibility.

mialy-defelice commented 2 years ago

@ychae updated this issue based on the work that I did yesterday to get the table up and running for NF.