crate / commons-codec

Data decoding, encoding, conversion, and translation utilities.
https://commons-codec.readthedocs.io
GNU Lesser General Public License v3.0
2 stars 2 forks source link

SQLParseException: Dynamic nested arrays are not supported #48

Open amotl opened 1 month ago

amotl commented 1 month ago

Problem

Nested arrays do not store well into CrateDB, right? Example:

[["150", "55"], null]

Thoughts

Over on this other issue from the other day, you shared valuable ideas how to work around the problem at hand.

Maybe you can come up with a good idea about this problem as well? 🙏

/cc @surister, @matriv, @proddata

seut commented 1 month ago

@amotl

Nested arrays are supported since CrateDB 5.7.0.

proddata commented 1 month ago

@seut I think @amotl is referring to not being able to add nested arrays to dynamic objects

INSERT INTO t01 (obj) VALUES({arr=[['150', '55'], null]});
-- SQLParseException[Dynamic nested arrays are not supported]

as documented here https://github.com/crate/crate/issues/15545

amotl commented 1 month ago

Most probably yes. Apologies that I did not specify my observations exactly enough within the original post. So, figuring from GH-15545, it is currently possible to store them, but only as top-level columns? I see, thanks.

However, we can't use it when following the data OBJECT(DYNAMIC) catch-all strategy we are employing on different ingest paths within I/O adapters to other databases, where we store the whole ingress record into a single column in CrateDB.

So, I've upvoted GH-15545. Thanks again!