search

gafusion / omas

Ordered Multidimensional Array Structure
http://gafusion.github.io/omas
MIT License
30 stars 14 forks source link

NetCDF Doesn't Support Logical/Boolean Variables so Saving Code Parameters Fails #288

Open torrinba opened 4 months ago

torrinba commented 4 months ago

This problem has been around for a while and can be avoided by saving with HDF5 instead, but I'm opening this issue to document the limitation and see if anyone has ideas about whether other solutions could be possible.

NetCDF doesn't support any logical or boolean variable types so an error gets thrown when there's one in the code.parameters:

TypeError: illegal primitive data type, must be one of dict_keys(['S1', 'i1', 'u1', 'i2', 'u2', 'i4', 'u4', 'i8', 'u8', 'f4', 'f8']), got bool

It would be possible for OMAS to convert these to integers {0,1} before saving to the file, but there wouldn't be any way to know if an integer stored in NetCDF should be converted to a logical variable when read in, since code.parameters doesn't have any defined structure.

orso82 commented 4 months ago

Perhaps for those variables you could convert the booleans to integers, and then mark the conversion with an attribute (something like .setncattr('is_boolean', 'true')). Then of course, you'll have to take care of that when reading the file back into OMAS.

smithsp commented 4 months ago

Note that in OMFIT we go through this rigamarole for exporting to netcdf - https://github.com/gafusion/OMFIT-source/blob/unstable/omfit/omfit_classes/omfit_data.py#L35 - then again when importing from netcdf - https://github.com/gafusion/OMFIT-source/blob/unstable/omfit/omfit_classes/omfit_data.py#L138 . Seems like an opportunity for generalizing the approach into a common set of utilities.

github-actions[bot] commented 2 months ago

Stale issue message