simonw / csv-diff

Python CLI tool and library for diffing CSV and JSON files
Apache License 2.0
292 stars 47 forks source link

BUG: incomplete last line causes unhandled error that's hard to debug #30

Open corneliusroemer opened 2 years ago

corneliusroemer commented 2 years ago

When I diffed two files I got csv-diff to give me this strange error:

ValueError: too many values to unpack (expected 2)

Turns out that the last line in on of the files being diffed is incomplete, it lacks a number of fields.

However, csv-diff doesn't handle the error as gracefully as it should. The user should be notified instead of getting a traceback.

Reproduce

printf "a,b,c,d\n1,2,3,4\n2,2,3,4\n3,2,3" >a.csv
printf "a,b,c,d\n1,2,3,4\n2,2,3,4\n3,2,3,4" >b.csv
csv-diff a.csv b.csv --key a
Traceback (most recent call last):
  File "/usr/local/Caskroom/mambaforge/base/bin/csv-diff", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/Caskroom/mambaforge/base/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/Caskroom/mambaforge/base/lib/python3.9/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/usr/local/Caskroom/mambaforge/base/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/Caskroom/mambaforge/base/lib/python3.9/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/Caskroom/mambaforge/base/lib/python3.9/site-packages/csv_diff/cli.py", line 60, in cli
    diff = compare(load(previous), load(current), show_unchanged)
  File "/usr/local/Caskroom/mambaforge/base/lib/python3.9/site-packages/csv_diff/__init__.py", line 86, in compare
    "changes": {
  File "/usr/local/Caskroom/mambaforge/base/lib/python3.9/site-packages/csv_diff/__init__.py", line 91, in <dictcomp>
    for _, field, (prev_value, current_value) in diffs
ValueError: not enough values to unpack (expected 2, got 1)