Add Scala & Python support to benchmarking

apache / datafusion-comet

Apache DataFusion Comet Spark Accelerator

https://datafusion.apache.org/comet

Apache License 2.0

823 stars 163 forks source link

Add Scala & Python support to benchmarking #967

Open holdenk opened 1 month ago

holdenk commented 1 month ago

What is the problem the feature request solves?

For folks considering adopting Arrow Datafusion Comet who have Dataset code in Scala & Python it would probably be useful to be able to run similar benchmarks to TPCDS but with Scala/Python code

Describe the potential solution

No response

Additional context

No response

andygrove commented 1 month ago

Thanks @holdenk. That would be great. Since you mentioned Python, you may be interested in https://github.com/apache/datafusion-comet/issues/957 as well.