search

deephaven / deephaven-core

Deephaven Community Core
Other
249 stars 79 forks source link

Support Flight DenseUnion type #5904

Open niloc132 opened 1 month ago

niloc132 commented 1 month ago

Prerequisite for https://github.com/deephaven/deephaven-core/issues/5339 to support CommandGetSqlInfo - the FlightSql spec calls for a "get sql info" call to be satisfied by returning a table with two columns, one uint32, and one dense union of several types.

Alternative short term workaround: update ArrowFlightUtil.DoExchangeMarshaller.SnapshotRequestRequestHandler.handleMessage() to have special handling for arrow tables, something vaguely like

                                    if (export instanceof VectorSchemaRoot) {
                                        // TODO implement serialization of VectorSchemaRoot to flight wire format
                                        return;
                                    }

Then, we could simply export a VectorSchemaRoot instead of a table, and any client can read it. This might confuse some users, since it would appear possible to take a "table ticket" and do other things do it (join it, filter it, etc), but that would fail for this type.

jmao-denver commented 1 month ago

Although the current unimplemented response for CommandGetSqlInfo causes the adbc-driver-flightsql to raise a warning during making the server connection, it doesn't seem to prevent it from executing ad-hoc SQL commands.