can add parquet as destination?

petl-developers / petl

Python Extract Transform and Load Tables of Data

MIT License

1.22k stars 190 forks source link

Open wonb168 opened 1 year ago

wonb168 commented 1 year ago

now have many destinations, ex. tocsv, todb... can add toparquet in next version?

juarezr commented 1 year ago

Maybe Apache Arrow could help.

wonb168 commented 1 year ago

I used connector-x to export pg table into arrow，but is so slowly，10* slower than csv export

blais commented 9 months ago

+1 to Parquet output. Really surprised it's not here.

juarezr commented 2 months ago

Parquet implementations available: