search

datajoint / datajoint-docs-original

https://docs.datajoint.org
Other
2 stars 13 forks source link

document floating point NaN/+Inf/-Inf serialization issues/workarounds #182

Open ixcat opened 5 years ago

ixcat commented 5 years ago

floating point serialization limitations/workarounds should be clearly documented with available workarounds if possible

dimitri-yatsenko commented 5 years ago

These are not serialization issues. This is when the data are not serialized. MySQL simply failed to implement the full IEEE 754 support for floating numbers, including infinities, signed zeros, and NaNs. We do not have a workaround for infinities and signed zeros. Just need to document this.

ixcat commented 4 years ago

from slack 2020-02-26:

The main workarounds are usually:
a) storing as numpy arrays (preserves all values but loses ability to directly query)
b) using ‘magic’ values contained in the code or a lookup table
c) using a flag field along the data value (e.g Foo table has foo and  foo_flag attributes)
ixcat commented 4 years ago

also use of null/nullable can be used for auto fillin NaN