search

xorbitsai / xorbits

Scalable Python DS & ML, in an API compatible & lightning fast way.
https://xorbits.io
Apache License 2.0
1.06k stars 67 forks source link

BUG: pd.read_csv(compression="gzip") can not run paralllel #738

Open luweizheng opened 9 months ago

luweizheng commented 9 months ago

Describe the bug

If we read a gz file with pd.read_csv(compression="gzip"), xorbits currently can only use 1 core and could not run parallel.