Open Cornelius-G opened 2 years ago
It seems to boil down to the missing support in JLD2 of flexible types in structs.
using ValueShapes
using JLD2
jldsave("test.jld2",x=NamedTupleShape{(:a,:b)})
jldsave("test.jld2",x=NamedTupleShape)
struct test{X<:(NTuple{N,Integer} where N)}
a::X
end
jldsave("test.jld2", a=test)
struct test_workaround
a::(NTuple{N,Integer} where N)
end
jldsave("test.jld2", a=test_workaround)
NamedTupleShapes frequently dispatches by NamedTupleShapes{names}, where N is left undefined.
Suggestion:
--- a/src/named_tuple_shape.jl
+++ b/src/named_tuple_shape.jl
@@ -37,14 +37,14 @@ all(x -> x == 4.2, view(flatview(data), 1, :))
"""
-struct NamedTupleShape{names,AT<:(NTuple{N,ValueAccessor} where N),VT} <: AbstractValueShape
- _accessors::NamedTuple{names,AT}
+struct NamedTupleShape{names,VT} <: AbstractValueShape
+ _accessors::NamedTuple{names,<:NTuple{N,ValueAccessor}} where N
For our project, the latter has a long rat tail reaching into BAT.jl. However I didn't see any direct usage of the length N information or the type AT anywhere. Will check if Zygote is affected.
Currently, it's not possible to save
ShapedAsNT
objects with JLD2. Minimal example:This gives the following stacktrace.txt
@waldie11 and I tried solving that by implementing
JLD2.writeas
andBase.convert
but didn't succeed.We think the problem is somehow related to
NamedTupleShape
being deconstructed into its components in "JLD2/src/data/writing_datatypes.jl" finally resulting in the error in the functionh5convert
. This only happens ifNamedTupleShape
isShapedAsNT
. Even though we tried for quite some time, we couldn't find a fix.We noticed that the problem is the same when trying to store a type with unknown size like this:
(for which we are not sure if something like that should even be possible)