search

apache / datafusion

Apache DataFusion SQL Query Engine
https://datafusion.apache.org/
Apache License 2.0
6.31k stars 1.19k forks source link

hooks for `temporary` tables in the catalog #12463

Closed hailelagi closed 1 month ago

hailelagi commented 2 months ago

Is your feature request related to a problem or challenge?

While trying to return an explicit error to clarify the semantics of temporary tables in #12363, it was suggested that it might be a good idea to semantically distinguish temporary tables in the catalog, while strictly speaking this behaviour is/already exists in that DF's tables support this behaviour.

Describe the solution you'd like

I would like to clearly define the semantics of a temporary table in the catalog.

Describe alternatives you've considered

No response

Additional context

No response

alamb commented 2 months ago

How about adding a flag on the appropriate DdlStatement

https://docs.rs/datafusion/latest/datafusion/logical_expr/enum.DdlStatement.html

So for example, that would mean adding a temporary flag to structures like CreateMemoryTable, CreateExternalTable, etc

https://docs.rs/datafusion/latest/datafusion/logical_expr/struct.CreateMemoryTable.html

If we wanted we could plumb that information into the memory catalog implementation that comes with DataFusion or we could potentially simply error if it appears (and let users of DataFusion define the semantics)

hailelagi commented 2 months ago

take