search

wireservice / csvkit

A suite of utilities for converting to and working with CSV, the king of tabular file formats.
https://csvkit.readthedocs.io
MIT License
6.03k stars 603 forks source link

csvsql Import to Mysql #1266

Open scoleri opened 1 month ago

scoleri commented 1 month ago

I've had to downgrade to 1.5.0 because with 2.0.1 i'm getting:

TypeError: conversion from NoneType to Decimal is not supported

This is when importing to mysql.

csvsql --db "mysql+mysqlconnector://admin:$PASS@$HOST:3306/blah" --table "TableBlah" --create-if-not-exists --unique-constraint ReportId,InstanceId --insert -
jpmckinney commented 1 month ago

I’ll need the CSV data to debug.

Gunbit commented 2 weeks ago

I have the same problem here, are the CSV data last_sync_at,number,description,description2,unit_price,vat_group,base_unit,net_weight,remaining_quantity 2024-11-06 14:47:29.077000,1000079510,ERDUNGSKABEL 120mm lg.,(1 Verpackungseinheit = 10 Stück),21.10000000000000000000,19,PACK,0.50000000000000000000,9.00000000000000000000 2024-11-06 14:47:29.077000,1000079610,ERDUNGSKABEL 230mm lg.,(1 Verpackungseinheit = 10 Stück),26.70000000000000000000,19,PACK,0.50000000000000000000,2.00000000000000000000 2024-11-06 14:47:29.077000,1000079705,ERDUNGSKABEL 330mm lg.,(1 Verpackungseinheit = 5 Stück),26.00000000000000000000,19,PACK,0.50000000000000000000,33.00000000000000000000 2024-11-06 14:47:29.077000,1000079805,ERDUNGSKABEL 430mm lg.,(1 Verpackungseinheit = 5 Stück),28.10000000000000000000,19,PACK,0.50000000000000000000,7.00000000000000000000

@jpmckinney thx

jpmckinney commented 2 weeks ago

I can't reproduce the issue. Please share the full command you ran, and csvkit version number as requested here https://github.com/wireservice/csvkit/blob/master/CONTRIBUTING.rst

Gunbit commented 2 weeks ago

It's happened sometimes version was 2.0.1. It does not currently happen with 2.0.0.

You will receive further information from me

sgpeter1 commented 2 weeks ago

According to the SQLAlchemy documentation:

The MySQL Connector/Python DBAPI has had many issues since its release, some of which may remain unresolved, and the mysqlconnector dialect is not tested as part of SQLAlchemy’s continuous integration. The recommended MySQL dialects are mysqlclient and PyMySQL.

I'm not sure if it'll solve your issue here, but perhaps consider using mysqlclient or PyMySQL.