search

microsoft / BatteryML

MIT License
497 stars 107 forks source link

[draft] Add CSV support #39

Closed HasanTheSyrian closed 4 months ago

HasanTheSyrian commented 4 months ago

Since this project aims for data to be accessible to everyone including non-programmers I thought adding support for exporting csv made sense.

But since there is no documentation for the codebase I have faced a bit of a challenge.

Right now, exporting works but with some datasets, there might be an extra/missing row/column. Each dataset has its own inputs and outputs as seen in batteryml/processes/preprocess_DATASET.py

For example, exporting CALCE leaves the temperature data blank since it doesn't exist. I'm not sure if it's supposed to be that way because the .pkl files are unreadable.

Another example; when exporting MATR there is an extra "Qdlin" row that isn't supposed to be part of output. It's part of the input.

The output rows are in each dataset's batteryml/processes/preprocess_DATASET.py but these files are not imported in battery_data.py where the output is shaped AFAIK.

HasanTheSyrian commented 4 months ago

@microsoft-github-policy-service agree

fingertap commented 4 months ago

Hi @HasanTheSyrian , thanks to your contribution! I am not sure I get your motivation. Do you recommend using csv is a more readable format for data storage?

For users with no programming expertise, we plan to implement GUI for BatteryML. For users with no programming capability, I am afraid that processing and visualize csv files can be still challenging.

HasanTheSyrian commented 4 months ago

Do you recommend using csv is a more readable format for data storage?

Yes.

I hope the repo will have documentation to encourage contribution. I'll close the PR for now.