JuliaIO / HDF5.jl

Save and load data in the HDF5 file format from Julia
https://juliaio.github.io/HDF5.jl
MIT License
390 stars 140 forks source link

Inconsistent writing of complex data inside compound type #1135

Open olof3 opened 9 months ago

olof3 commented 9 months ago

Complex numbers are written differently if they are inside a compound type compared to pure complex data.

complex_data = [1+1im, 2+1im]
namedtuple_data = [(z=z, index=k) for (k,z) in enumerate(complex_data)]

println("Original complex_data: ", complex_data)
println("Original namedtuple_data: ", namedtuple_data)

h5open("test.h5", "w") do file
    file["complex_data"] = complex_data
    file["namedtuple_data"] = namedtuple_data
end

h5open("test.h5", "r") do file
    println("Written & read complex_data: ", read(file["complex_data"]))
    println("Written & read namedtuple_data: ", read(file["namedtuple_data"]))
end

gives

Original complex_data: Complex{Int64}[1 + 1im, 2 + 1im]
Original namedtuple_data: NamedTuple{(:z, :index), Tuple{Complex{Int64}, Int64}}[(z = 1 + 1im, index = 1), (z = 2 + 1im, index = 2)]

Written & read complex_data: Complex{Int64}[1 + 1im, 2 + 1im]
Written & read namedtuple_data: NamedTuple{(:z, :index), Tuple{NamedTuple{(:re, :im), Tuple{Int64, Int64}}, Int64}}[(z = (re = 1, im = 1), index = 1), (z = (re = 2, im = 1), index = 2)]

With

HDF5.set_complex_field_names("re", "im")

the data that is read back has the same format as what was written,

Original complex_data: Complex{Int64}[1 + 1im, 2 + 1im]
Original namedtuple_data: NamedTuple{(:z, :index), Tuple{Complex{Int64}, Int64}}[(z = 1 + 1im, index = 1), (z = 2 + 1im, index = 2)]

Written & read complex_data: Complex{Int64}[1 + 1im, 2 + 1im]
Written & read namedtuple_data: NamedTuple{(:z, :index), Tuple{Complex{Int64}, Int64}}[(z = 1 + 1im, index = 1), (z = 2 + 1im, index = 2)]

Perhaps this indicates that the inconsistency is due to that the special treatment in #558 (which reads/writes complex numbers in hdf5 files as compound types with fields r and i) is applied to nested complex when they are read but not when they are written?

mkitti commented 9 months ago

Indeed, Complex support preceded the mapping between named tuples and compound types. I'm not sure that Iike that we have special treatment of Complex.

My preference here would be for the user to explicitly supply the type as the second parameter to read. This is better in terms of type stability.