simonw / csv-diff

Python CLI tool and library for diffing CSV and JSON files
Apache License 2.0
292 stars 47 forks source link

KeyError in load_csv #26

Closed dvm579 closed 2 years ago

dvm579 commented 2 years ago

Hello,

I am getting a KeyError when comparing two CSV's with the following header rows, using key='''"Event ID"''':

current.csv "","Event ID","Primary","EVENT DATE","Invoice #","5 Days Notice","Registration Links","Did the vendor provide the required staff level?","Tent, including anchoring device","Tables and chairs","Yoga mats/cots/stretchers/floor mats","Administrative supplies","Traffic control devices, if applicable","Food and fluids","Emergency supply bag (confirm with SO)","Fan, air conditioner","Mobile handwashing station","At least one sign more than 3 ft tall","Did all vendor staff arrive on time?","Was ready prior to start time?","3 EpiPens (at least 1 adult and 1 pediatric dose)","Did start on time?","Onsite entire time","Were there any avoidable patient care errors?","LNO Name","Have vendor representative enter their name","Vendor Comments","Admin Comments/Response"

update.csv "","Event ID","Primary","EVENT DATE","Invoice #","5 Days Notice","Registration Links","Did the vendor provide the required staff level?","Tent, including anchoring device","Tables and chairs","Yoga mats/cots/stretchers/floor mats","Administrative supplies","Traffic control devices, if applicable","Food and fluids","Emergency supply bag (confirm with SO)","Fan, air conditioner","Mobile handwashing station","At least one sign more than 3 ft tall","Did all vendor staff arrive on time?","Was ready prior to start time?","3 EpiPens (at least 1 adult and 1 pediatric dose)","Did start on time?","Onsite entire time","Were there any avoidable patient care errors?","LNO Name","Have vendor representative enter their name","Vendor Comments","Admin Comments/Response"

As you can see, the 2 header rows contain the same exact columns, and '"Event ID"' is present in both. However, the load_csv function is still returning: KeyError: '"Event ID"'

Any help would be greatly appreciated!

dvm579 commented 2 years ago

Here is the full stacktrace of the error when I run it using the CLI and the same files:

Traceback (most recent call last): File "/Users/dhruvamagar/PycharmProjects/IEMAFeedbackScraper/venv/bin/csv-diff", line 8, in sys.exit(cli()) File "/Users/dhruvamagar/PycharmProjects/IEMAFeedbackScraper/venv/lib/python3.9/site-packages/click/core.py", line 1128, in call return self.main(args, kwargs) File "/Users/dhruvamagar/PycharmProjects/IEMAFeedbackScraper/venv/lib/python3.9/site-packages/click/core.py", line 1053, in main rv = self.invoke(ctx) File "/Users/dhruvamagar/PycharmProjects/IEMAFeedbackScraper/venv/lib/python3.9/site-packages/click/core.py", line 1395, in invoke return ctx.invoke(self.callback, ctx.params) File "/Users/dhruvamagar/PycharmProjects/IEMAFeedbackScraper/venv/lib/python3.9/site-packages/click/core.py", line 754, in invoke return __callback(args, **kwargs) File "/Users/dhruvamagar/PycharmProjects/IEMAFeedbackScraper/venv/lib/python3.9/site-packages/csv_diff/cli.py", line 60, in cli diff = compare(load(previous), load(current), show_unchanged) File "/Users/dhruvamagar/PycharmProjects/IEMAFeedbackScraper/venv/lib/python3.9/site-packages/csv_diff/cli.py", line 56, in load return load_csv( File "/Users/dhruvamagar/PycharmProjects/IEMAFeedbackScraper/venv/lib/python3.9/site-packages/csv_diff/init.py", line 26, in load_csv return {keyfn(r): r for r in rows} File "/Users/dhruvamagar/PycharmProjects/IEMAFeedbackScraper/venv/lib/python3.9/site-packages/csv_diff/init.py", line 26, in return {keyfn(r): r for r in rows} File "/Users/dhruvamagar/PycharmProjects/IEMAFeedbackScraper/venv/lib/python3.9/site-packages/csv_diff/init.py", line 21, in keyfn = lambda r: r[key] KeyError: 'Event ID'

dvm579 commented 2 years ago

I just figured it out, I had to add dialect="excel"

These csv files are being generated by pandas.to_csv with no additional parameters, if that is useful for anybody.