posit-dev / positron

Positron, a next-generation data science IDE
https://positron.posit.co
Other
2.83k stars 91 forks source link

Add support for compressed CSV/TSV file extensions when opening in data explorer #5332

Open wesm opened 1 week ago

wesm commented 1 week ago

DuckDB will automatically use gzip/zstd to decompress files with a standard extension like .csv.gz, so we can easily add support for opening these in the data explorer. Based on the size of the file, we may choose to use CREATE TABLE vs. CREATE VIEW to avoid every subsequent query having to decompress the file, which will degrade latency and increase CPU utilization when using the data explorer.