search

apache / datafusion-comet

Apache DataFusion Comet Spark Accelerator
https://datafusion.apache.org/comet
Apache License 2.0
798 stars 155 forks source link

Implement native parsing of CSV files #882

Open andygrove opened 1 month ago

andygrove commented 1 month ago

What is the problem the feature request solves?

We can probably accelerate reading of CSV files by continuing to use JVM Spark to read bytes from disk but then parse the CSV in native code.

Describe the potential solution

No response

Additional context

No response

psvri commented 1 month ago

Hello.

I would like to start working on this.