ArrayType serialization with 2D+ arrays

[Byrth]

Currently ArrayType() iterates element-wise over an array type using the default iterate(x::T) method when serializing. This has the effect of flattening higher dimensional arrays into vectors. Matrices, for instance, become vectors with no information about their original dimensions.

The JSON standard maintains array order, so it seems safe for me to serialize matrices as vectors of vectors. However, my only way to affect serialization is by overloading the iterate(x::T) function and doing so for the Matrix type seems likely to recompile large parts of Julia base (as well as break my own code.)

Then on deserialization, I would just construct(::Type{Matrix}, x::Vector) = hcat(x...)

It makes sense on some level that this would be natively unsupported because it makes the serialized representation of matrices and vectors of vectors indistinguishable (and thus deserialize(serialize(matrix)) != matrix without an additional construct definition), but that code doesn't work right now either and it seems like there should be some way to for users to enable support for it because matrices are so common. Perhaps there should be something analogous to construct() for serialization?

[tpapp]

I agree that it would be great to support this natively. A lot of formats (JSON, TOML, ...) just support flat vectors.

I am wondering if <: AbstractArray types except <: AbstractVector could be serialized as a StructTypes.Struct() instead, with the axes and the elements (as a vector) as fields. Perhaps even rename the current ArrayType to VectorType.

[disberd]

Would this feature also inherently enable multidimensional array support in JSON3? https://github.com/quinnj/JSON3.jl/issues/196

[quinnj]

Yeah, we could perhaps explore implementing @tpapp's idea of serializing higher-order arrays as a struct and see how that goes.