larsyencken / csvdiff

Generate a diff between two tabular datasets expressed in CSV files.
BSD 3-Clause "New" or "Revised" License
132 stars 31 forks source link

Export to xlsx using XlsxWriter #18

Closed ivansabik closed 5 years ago

ivansabik commented 8 years ago

Would be nice to have support export to XLS, csv or plain DF like this, I'll try to contribute to that myself:

untitled
ivansabik commented 8 years ago

Something like this worked for me:

from urllib2 import Request, urlopen
import json
import pandas as pd
from pandas.io.json import json_normalize

with open('tmp/output_csvdiff.json') as data_file:
    data = json.load(data_file)

added = json_normalize(data['added']) if data['added'] else pd.DataFrame(['No records added'])
removed = json_normalize(data['removed']) if data['removed'] else pd.DataFrame(['No records removed'])
changed = json_normalize(data['changed']) if data['changed'] else pd.DataFrame(['No records changed'])

# Write to Excel
writer = pd.ExcelWriter('/tmp/output_csvdiff.xlsx')

added.to_excel(writer, sheet_name='Added', index=False)
removed.to_excel(writer, sheet_name='Removed', index=False)
changed.to_excel(writer, sheet_name='Changed', index=False)

writer.save()

This will write an Excel file with three sheets:

inforce_diff
ivansabik commented 8 years ago

Just found this one, so maybe only XLSX:

https://github.com/larsyencken/csvdiff/blob/master/tests/test_csvdiff.py#L372

larsyencken commented 8 years ago

Sure, a contribution would be most welcome!

The example you found was actually for saving records as CSV, but you want to create a way to save the patch format as CSV (and then XLSX). Once you have the columns you're going to use, the rest should fall out. I'd start with the key columns, then have the from and to versions of any column that had changed values, as you suggested.

With XLSX export, I've tried to avoid having too many dependencies, in particular pandas because it's heavy. But, I'd be happy to support it as a format with an optional dependency. If you have it installed, the export works, otherwise you get an error message telling you what to install. Sound ok?

ivansabik commented 8 years ago

Sound great, I'll work on something and will send a PR then great tool Lars!

ivansabik commented 8 years ago

It's done, still need to add those tests though: https://github.com/ivansabik/csvdiff/tree/xls_output