-
# Call to action:
Let's invest more effort in DataFusion benchmarking, both as a mechanism for technical evangelism as well as a guide for actual performance improvements.
# Background
We ha…
-
### What is the problem the feature request solves?
### What is the problem the feature request solves?
We currently delegate to DataFusion when casting from string to decimal and there are some d…
-
I know this is a work in progress, I just wanted to say that I think it's great!
I am sure you've seen https://josiahparry.com/posts/2023-11-24-dfusionrdr/#source-code , but wanted to put it here i…
-
### What is the problem the feature request solves?
I'm running the 1TB TPCDS benchmark over Comet and Vanilla Spark.
I'm running on a 48Core 186G RAM machine
Here's my config:
```
/localhdd/…
-
### What is the problem the feature request solves?
From [planga82](https://github.com/planga82)'s comment: https://github.com/apache/datafusion-comet/pull/355#issuecomment-2086056132
> As an idea…
-
Previous discussion: https://github.com/apache/arrow-datafusion/issues/4707
Though the ORC format is not as widely used as parquet in arrow-rs and datafusion related projects, there are still some …
-
# Description
The rust writer in it current state keeps a buffer instead of steaming to disk which causes the writer use quite some extra memory.
We need to address this performance issue.
@wjones1…
-
### Is your feature request related to a problem or challenge?
As @goldmedal started trying to move the DynamicFileProvider so others could use it in https://github.com/apache/datafusion/pull/10745…
alamb updated
2 weeks ago
-
### Is your feature request related to a problem or challenge?
We have an example for writing a user defined optimizer rule in
https://github.com/apache/datafusion/blob/3773fb7fb54419f889e7d18b73…
-
### Is your feature request related to a problem or challenge?
As we work to make extracting statistics from parquet data pages more correct and performant in https://github.com/apache/datafusion/is…