wireservice / csvkit

A suite of utilities for converting to and working with CSV, the king of tabular file formats.
https://csvkit.readthedocs.io
MIT License
6.03k stars 603 forks source link

csvsql - TypeError: conversion from NoneType to Decimal is not supported #1258

Closed clach04 closed 3 months ago

clach04 commented 3 months ago

When using csvsql against either mysql or ingres get error with simple csv file:

Contents of file bug_two_columns.csv

col1,col2
data,1

Demo, note this uses Ingres https://github.com/ActianCorp/sqlalchemy-ingres but mysql should fail in exact same way:

(py3venv) C:\code\py\csvkit_agatesql>csvsql --db ingres:///csvsql_bug --tables csvsql_bug --insert bug_two_columns.csv
TypeError: conversion from NoneType to Decimal is not supported

Workaround is to manually and explicitly specify lengths for two parameters:

csvsql --db ingres:///csvsql_bug --tables csvsql_bug --insert bug_two_columns.csv  --col-len-multiplier=1 --min-col-len=1

There are arguably two causes (possible fix points) of this bug:

  1. csvsql has bad defaults "None" for these parameters (col-len-multiplier and --min-col-len) - PR posted https://github.com/wireservice/csvkit/pull/1259
  2. https://github.com/wireservice/agate-sql has no validation checks - https://github.com/wireservice/agate-sql/issues/46
clach04 commented 3 months ago

Pull request https://github.com/wireservice/csvkit/pull/1259 posted/linked.

jpmckinney commented 3 months ago

Fixed by 5ba0aa7. Thank you for reporting!