cannot read old CSV files

[anarcat]

for some reason, this software cannot read the files created from the older "battery-status" version. i get this error:

$ /usr/share/battery-stats/battery-stats-graph /var/log/hjemmenett-battery-status.log
loading CSV file <open file '/var/log/hjemmenett-battery-status.log', mode 'r' at 0x7fba9dc6e420> with builtin CSV module
Traceback (most recent call last):
  File "/usr/share/battery-stats/battery-stats-graph", line 171, in <module>
    data = parse_csv()
  File "/usr/share/battery-stats/battery-stats-graph", line 74, in parse_csv_builtin
    if '' == row[f]:
KeyError: 'charge_now'

is there a way to convert the files over? my original python version still works:

$ ~/src/battery-status/battery-status-graph.py /var/log/hjemmenett-battery-status.log
loading CSV file <open file '/var/log/hjemmenett-battery-status.log', mode 'r' at 0x7fe9238b6420> with builtin CSV module
building data array
guessing expiry
this battery will reach end of life (5%) in 904 days, 23:22:16.832222, on 2018-09-15 16:08:27.770046
building graph
drawing on tty

... and crap, it looks like my battery is already at 80% effective capacity!

[anarcat]

after comparing the headers, here is what changed:

timestamp 
[when]
manufacturer 
model_name 
technology 
serial_number 
[status]
[cycle_count]
[energy_now]
energy_full 
energy_full_design 
[charge_now]
[charge_full]
[charge_full_design]

all the fields in brackets have either been added or moved. no field was removed.

by using Gnumeric (or any spreadsheet), I was able to load this up as a regular CSV file, and move the stuff around. the basic human algorithm is:

• insert column after column timestamp (A), name it when
• in cell B2, insert the function =unix2date(A2), format as a ISO timestamp (Format, Cells, Format, Number, Date, yyyy-mm-dd"T"hh:mm:ss"Z", we're ignoring timezone offsets for now
• insert three columns after serial_number (now column F)
• cut and paste columns status, cycle_count and energy_now in columns G, H and I, respectively
• add three headers for charge_now, charge_full and charge_full_design

Then in gnumeric, the export is well hidden in Data, Export data...

It would be great to have an automated way to convert old files, but since it was just my use case, i didn't bother.. :/

actually, that doesn't work: the when column format is different from the one generated by the scripts, so the parser freaks out when new entries are added. libreoffice and gnumeric both export timestamps as YYYY-MM-DD HH:MM:SS whereas the scripts expects an ISO date. no idea how to change the export format. :( damn spreadsheets.

update: scratch that, the format changes in the spreadsheet actually affect the CSV output format in libreoffice (but not gnumeric), so i was able to make generate a fairly similar file.

[petterreinholdtsen]

I believe the commit 46354240a88e7dc99236536a58d2621cd7446424 solved this issue, as the graph script now work also when the charge* fields are missing.

[anarcat]

well the problem here is that the column order is different: if you take an old logfile and append to it with the new scripts, you get the wrong headers for parts of the data.

[petterreinholdtsen]

Sure. This is the reason I changed the file name. While it would be nice to be able to migrate the data, I believe it is fundamentally impossible as two possible values were stored in one field. But if it is possible, I would be happy to accept a pull request with a migration tool.