apache / datafusion

Apache DataFusion SQL Query Engine
https://datafusion.apache.org/
Apache License 2.0
6.27k stars 1.18k forks source link

Support adding a single new table factory to `SessionStateBuilder` #12552

Closed alamb closed 1 month ago

alamb commented 1 month ago

Is your feature request related to a problem or challenge?

Part of https://github.com/apache/datafusion/issues/12550

While working on https://github.com/datafusion-contrib/datafusion-dft I want to register various types of extensions while configuring the SessionContext, ideally adding each extension to the SessionStateBuilder each time.

However, I found that there were a few APIs missing on SessionStateBuilder so I had to implement my own workaround builder here: https://github.com/datafusion-contrib/datafusion-dft/blob/8247555f9464058c1ac3370196739ac2b19343ee/src/extensions/builder.rs#L84-L98

SessionStateBuilder has SessionStateBuilder::with_table_factories method to set all the table factories, but no way to just append a new table factory

Describe the solution you'd like

I would like a way to append just a single new TableFactory to the list of factories being constructed

Also it should have

  1. Documentation
  2. Tests (ideally a doc test with an example of how to use to use it)

Describe alternatives you've considered

I recommend adding a new function SessionStateBuilder::with_table_factory that works like this:

let state = SessionStateBuilder::new() 
  .with_table_factory("DELTA", factory1)
  .wth_table_facotry("ICEBERG", facotry2)
  // resulting state has both factory1 and factory2
  .build()

Additional context

No response

alamb commented 1 month ago

I think this is a good first issue as it is clearly described and straightforward to implement. It would be a good introduction to DataFusion I think

Weijun-H commented 1 month ago

\take

alamb commented 1 month ago

\take

BTW since you are now a committer, I think you can also just assign github issues directly to yourself :)