CerebusOSS / ella

A streaming time-series datastore for low-latency applications
https://docs.rs/ella/
Apache License 2.0
2 stars 0 forks source link

Information schema tables can't be accessed from remote clients #6

Open sydduckworth opened 1 year ago

sydduckworth commented 1 year ago

As part of logical plan serialization the engine makes the assumption that each table is either an EllaTable or a StubTable. This is done because the extension codec uses the full table ID during serialization, which isn't available for arbitrary tables.

This assumption holds generally since obviously all tables are created by the engine itself. The one exception is information_schema, which is auto-generated by datafusion, and uses the StreamingTable type. This causes an error when ella tries to serialize a logical plan that scans one of those tables.

A simple workaround would be to just add a custom wrapper type around datafusion's InformationSchemaProvider type that implements SchemaProvider by just wrapping returned tables in a second custom wrapper type that delegates to the contained table but exposes the table ID. Then the extension codec can just also check for that custom wrapper type during serialization.