search

jqnatividad / qsv

Blazing-fast Data-Wrangling toolkit
https://qsv.dathere.com
The Unlicense
2.52k stars 71 forks source link

`sqlp`: derive polars schema from stats cache #2256

Closed jqnatividad closed 3 weeks ago

jqnatividad commented 3 weeks ago

skipping the need for Polars to scan the CSV for --infer-len rows to guess the schema.

The guess may turn out to be wrong as the datatype inferences are not guaranteed as it only scans the first --infer-len rows, so this not only makes sqlp faster, it makes it more reliable.

resolves #2225