astrodbtoolkit / astrodb_utils

https://astrodb-utils.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
0 stars 4 forks source link

Add units expectations to ingest functions #20

Open kelle opened 9 months ago

kelle commented 9 months ago

In SIMPLE, we assume units and note what the expectation is in the docs. This is extremely error prone. Other options

arjunsavel commented 8 months ago

I think including in the column name is a bit more cumbersome, but especially in light of the Chen et al. 2022 recommendations, it makes sense to explicitly state units in columns.

Provide a clear title and unambiguous labels of columns. Indicate the units for each column when applicable. This is especially important when data are to be compared, or used outside a specialized field.

Thoughts?

kelle commented 8 months ago

I'm 50-50.

I'm leaning towards a dedicated units column. It's more flexible and makes things more interoperable. Also, easy to test. Harder to mess up since units have to be set deliberately.

kelle commented 8 months ago

Option: modify ingest scripts to expect Quantities and then the ingest functions convert to the desired unit. Have ignore_units option.

kelle commented 8 months ago

David doesn't like the idea of a units column. He is ok with having units in the column name.

arjunsavel commented 8 months ago

Not fully addressed yet - in a future PR still need to explicitly handle quantities on ingest.

arjunsavel commented 8 months ago

@kelle @dr-rodriguez for handling quantities on ingest...is that a feature we want to implement here or astrodb_scripts?

kelle commented 8 months ago

astrodb_scripts. I can move the issue.