wireservice / csvkit

A suite of utilities for converting to and working with CSV, the king of tabular file formats.
https://csvkit.readthedocs.io
MIT License
6.01k stars 603 forks source link

TypeError generating schema with csvsql #154

Closed ashaw closed 12 years ago

ashaw commented 12 years ago

I'm getting a TypeError trying to use csvsql to generate a table insert statement from a CSV

Here's the command:

head -n 2 expenditures-csv.csv | csvsql --no-constraints --table campaignexpenditures -v

Traceback:

  Traceback (most recent call last):
File "/usr/local/bin/csvsql", line 6, in <module>
  CSVSQL().main()
File "/Library/Python/2.6/site-packages/csvkit/utilities/csvsql.py", line 43, in main
  csv_table = table.Table.from_csv(self.args.file, name=table_name, snifflimit=self.args.snifflimit, **self.reader_kwargs)
File "/Library/Python/2.6/site-packages/csvkit/table.py", line 221, in from_csv
  columns.append(Column(column_ids[i], headers[i], c))
File "/Library/Python/2.6/site-packages/csvkit/table.py", line 31, in __init__
  t, data = typeinference.normalize_column_type(l)
File "/Library/Python/2.6/site-packages/csvkit/typeinference.py", line 133, in normalize_column_type
  d = parse(x, default=DEFAULT_DATETIME)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/dateutil/parser.py", line 700, in parse
  return DEFAULTPARSER.parse(timestr, **kwargs)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/dateutil/parser.py", line 300, in parse
  res = self._parse(timestr, **kwargs)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/dateutil/parser.py", line 560, in _parse
  res.hour += 12
TypeError: unsupported operand type(s) for +=: 'NoneType' and 'int'

On this data:

committee,committee_name,election_other_description,date_signed,support_oppose_code,payee_zip,payee_street_1,candidate_state,completing_last_name,payee_street_2,entity_type,candidate_office,filer_committee_id_number,candidate_first_name,candidate_prefix,expenditure_amount,payee_city,candidate_suffix,candidate_last_name,payee_organization_name,payee_cmtte_fec_id_number,candidate_middle_name,completing_first_name,completing_middle_name,transaction_id_number,category_code,payee_first_name,memo_text_description,payee_state,completing_suffix,completing_prefix,expenditure_purpose_descrip,expenditure_date,back_reference_tran_id_number,candidate_id_number,memo_code,back_reference_sched_name,payee_prefix,payee_last_name,candidate_district,calendar_y_t_d_per_election_office,payee_middle_name,form_type,payee_suffix,election_code
C00431445,OBAMA FOR AMERICA,,,,94160,"File 74009, PO Box #60000",,,,ORG,,C00431445,,,17826.22,San Francisco,,,CyberSource Corporation ,,,,,SB23-117180,,,,CA,,,Credit Card Fees ,20120103,,,,,,,,,,SB23,,P2012
onyxfish commented 12 years ago

This was actually very recently fixed on master (commit 557a5e90ecb7d7b5a9c9c85b00ea6ac3a2749102) though there was never a ticket for it. I just tested with your file and hit a different bug, but I've fixed that as well.