search

nv-legate / legate

The Foundation for All Legate Libraries
https://docs.nvidia.com/legate
193 stars 63 forks source link

Legate.io: "unsupported type code: binary" #961

Open manopapad opened 2 weeks ago

manopapad commented 2 weeks ago

Slac code gets error when reading their dataset:

Task "legate::experimental::io::hdf5::detail::HDF5Read" threw an unsupported exception type (std::runtime_error): "unsupported type code: binary".

simple change like below didn't fix it:

diff --git a/src/cpp/legate/experimental/io/hdf5/interface.cc b/src/cpp/legate/experimental/io/hdf5/interface.cc
index ab7dd8c2..121f189f 100644
--- a/src/cpp/legate/experimental/io/hdf5/interface.cc
+++ b/src/cpp/legate/experimental/io/hdf5/interface.cc
@@ -82,11 +82,12 @@ namespace {
     }
     case HighFive::DataTypeClass::BitField:
       return legate::binary_type(static_cast<std::uint32_t>(dtype.getSize()));
+    case HighFive::DataTypeClass::Opaque:
+      return legate::binary_type(static_cast<std::uint32_t>(dtype.getSize()));
     case HighFive::DataTypeClass::String: return legate::string_type();
manopapad commented 1 week ago

The underlying issue is that HDF5 doesn't have a native boolean datatype. Booleans are stored as uint8 datasets. The best workaround for now is to just parse it as a uint8 datatype, and cast it using https://github.com/nv-legate/legate.core/issues/963 to a boolean Store.

manopapad commented 1 week ago

Retargeting this nominally for 25.01, but will try to make it available through weekly drop ASAP.