Replace `NaN` in protein metadata

KarrLab / datanator_rest_api

A OAS3 compliant REST API for the Datanator integrated database

MIT License

0 stars 3 forks source link

Replace `NaN` in protein metadata #110

Closed jonrkarr closed 4 years ago

jonrkarr commented 4 years ago

Example: https://testapi.datanator.info/proteins/meta/meta_combo/?uniprot_id=P37173

jonrkarr commented 4 years ago

I changed the frontend to use JSON5 so that it won't fail to parse JSON strings from the API when they include NaN.

To fully take advantage of this, everywhere the frontend detect null it should check null or NaN. Because this applies to many places in the code (and it would take some work to update this), it would still be helpful to replace this NaN with null on the backend.

lzy7071 commented 4 years ago

Agreed. My temporary solution is also essentially converting NaN to null on the backed. The difficulty lies in the fact that the fields that contain NaN are not constant. I might have to iterate through all the 2.8M documents in uniprot collection.

jonrkarr commented 4 years ago

Is it always NaN or sometimes "NaN"? The former will be handled by JSON5.

lzy7071 commented 4 years ago

Is it always NaN or sometimes "NaN"? The former will be handled by JSON5.

Fairly certain it's NaN, as in numpy.nan, since they were generated by dataframe.

jonrkarr commented 4 years ago

If you know which collections/fields this is limited to, I can check for NaN where needed on the frontend. Then it won't be so important to fix this in the database.

lzy7071 commented 4 years ago

halflives object in rna_halflife, affected endpoints:

/rna/halflife/get_info_by_name
/rna/halflife/get_info_by_ko (Both of them are temporarily fixed, but not on the database side)

modifications object in uniprot collection, affected endpoints include anything that doesn't project out modifications object:

/protein/meta_combo/*
/protein/proximity_abundance/proximity_abundance_kegg
/protein/precise_abundance (temporarily fixed)

jonrkarr commented 4 years ago

This is now fully handled in the frontend.

I used JSON5 to handle NaN
All NaN are automatically converted to null

As a result, we no longer need to fix this in the backend.

Going forward, if the backend needs to communicate NaN (as different from null), we will need to remove (2) above and more carefully interpret NaN vs null throughout the frontend code.

lzy7071 commented 4 years ago

Awesome! Thank you!