Given that np.nan is a floating-point number -- unlike with ints/strings where Pandas allows you to stick floating-point NaN into a column containing otherwise int/string -- numpy.nan actually is a floating-point number. (This, along with Inf, is part of the IEEE floating-point specification).
Given that we have some refactoring going on with pushing more logic down to C++, code-deduping from Python & R -- which can have minor effects
As of tiledbsoma 1.9: I verified that if you ingested a floating-point column with NaN in it, that came in as float32 or float64 , and non-nullable
As of tiledbsoma 1.10 and 1.11: I verified that if you attempted to ingest a floating-point column with NaN in it, that failed with Casting field 'new_col' with null values to non-nullable
Thus it's possible that if a user had an experiment created with 1.9 or earlier which did have NaNs in a floating-point column, they would still see today Casting field 'new_col' with null values to non-nullable on an attempt to do tiledbsoma.io.update_obs with the new_col having any NaNs in it.
Marked as blocker since I want this in 1.12.0 (#2672)
Problem:
tiledbsoma.io.update_obs
or append-mode ingest data which does have either all nulls, or a mix of null and non-nulltiledbsoma.io.update_obs
or append-mode ingest fails withCasting field 'new_col' with null values to non-nullable
Side note:
numpy.nan
actually is a floating-point number. (This, along withInf
, is part of the IEEE floating-point specification).Casting field 'new_col' with null values to non-nullable
on an attempt to dotiledbsoma.io.update_obs
with thenew_col
having any NaNs in it.Marked as
blocker
since I want this in 1.12.0 (#2672)[sc-49403]