search

simonw / csvs-to-sqlite

Convert CSV files into a SQLite database
Apache License 2.0
879 stars 69 forks source link

error_bad_lines argument has been deprecated #88

Open frafra opened 2 years ago

frafra commented 2 years ago

I get a deprecation warning on the latest stable.

csvs_to_sqlite/utils.py:38: FutureWarning: The error_bad_lines argument has been deprecated and will be removed in a future version. Use on_bad_lines in the future.
LuighiV commented 1 year ago

Hi, This is giving an error now:

Could not load file.csv: read_csv() got an unexpected keyword argument 'error_bad_lines'

It would be great if it can be updated.

LuighiV commented 1 year ago

From the pandas release notes for the latest version (April, 3 2023)

Removed arguments prefix, squeeze, error_bad_lines and warn_bad_lines from read_csv() (GH40413, GH43427)

https://pandas.pydata.org/docs/whatsnew/v2.0.0.html#removal-of-prior-version-deprecations-changes

ross-spencer commented 1 year ago

Seeing this error in 1.3 now. A workaround is to temporarily roll back pandas: pip install 'pandas==1.4.0'

ribtoks commented 1 year ago

Also having this issue. Crazy that it's almost a year now! Maybe nobody is using csvs-to-sqlite?

virtadpt commented 1 year ago

Yep, that did it.

rtanglao commented 1 year ago

this worked for me too!

pip3 install 'pandas==1.4.0'
# git clone from this repo and cd to that directory
pip3 install -e .
panique commented 11 months ago

Also having this issue. Crazy that it's almost a year now! Maybe nobody is using csvs-to-sqlite?

Asked myself the same, but the tool has hundreds of downloads each WEEK! Looks like lots of people try it out and then give up as it doesnt work out of the box.

virtadpt commented 11 months ago

@panique Or we're just using it every day and don't see a need to mention it. It's just another tool in the toolbox, like awk or vi.

BryantD commented 11 months ago

If you've been using it all along and you haven't updated pandas, you won't see the issue -- this is a problem that occurs if you do a fresh install.

I'm working around it by just using sqlite-utils:

sqlite-utils insert reviews.db reviews reviews.csv --csv

is functionally identical to:

csvs-to-sqlite reviews.csv reviews.db

https://github.com/simonw/csvs-to-sqlite/issues/69 seems relevant here.

bwingen1 commented 4 months ago

If anyone's struggling with csvs-to-sqlite, this set of requirements works: pip>=24.2 csvs-to-sqlite datasette numpy==1.23.2 pandas==1.4.4

(Did this with py 3.9.6)

Esonhugh commented 3 months ago

If anyone's struggling with csvs-to-sqlite, this set of requirements works: pip>=24.2 csvs-to-sqlite datasette numpy==1.23.2 pandas==1.4.4

(Did this with py 3.9.6)

pipx user fixer

pipx inject csvs-to-sqlite pandas==1.4.4 --force
pipx inject csvs-to-sqlite numpy==1.23.2 --force