search

astrodbtoolkit / astrodb-template-db

A template for astronomical databases.
https://astrodb-template-db.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
1 stars 2 forks source link

Keep parallax error sig figs >= parallax sig figs #67

Closed arjunsavel closed 1 month ago

arjunsavel commented 3 months ago

Ideally, we don't want the number of significant figures in a quantity to exceed the precision established by our error bars. This PR enforces as much for the Parallax table.

dr-rodriguez commented 3 months ago

No description provided.

May want to actually describe what you are trying to do in the PR, beyond just the title

dr-rodriguez commented 3 months ago

From what I can tell, what you're trying to do is to enforce that if an error has 2 significant figures, then the value should as well. In practice, this is pretty good. However, databases and python variables are not always exact. There can be float rounding/truncation errors as values are moved from python to json to database and viceversa, which can cause ordinary values to suddenly have errors.

That said, I think most of our cases should be fine as I think float precision issues start to come up with very large/very small numbers. Things like parallax could be fine.

Would this be something better placed in the schema itself rather than in the database tests? The former means any new value must have matching significant figures, the later means that this is only examined after its ingested.

arjunsavel commented 3 months ago

These are great points! I'll try putting this in the schema.

arjunsavel commented 2 months ago

check: is this adding digits? what about sdr all the numbers?

arjunsavel commented 2 months ago

how to determine significant figures when 1.01 is stored as 1.00999..

kelle commented 2 months ago

check out https://stackoverflow.com/questions/8101353/counting-significant-figures-in-python

arjunsavel commented 2 months ago

have this as a validator WARNING for parallax error.