search

apache / datafusion-comet

Apache DataFusion Comet Spark Accelerator
https://datafusion.apache.org/comet
Apache License 2.0
825 stars 164 forks source link

Implement native parsing of CSV files #882

Open andygrove opened 2 months ago

andygrove commented 2 months ago

What is the problem the feature request solves?

We can probably accelerate reading of CSV files by continuing to use JVM Spark to read bytes from disk but then parse the CSV in native code.

Describe the potential solution

No response

Additional context

No response

psvri commented 2 months ago

Hello.

I would like to start working on this.