Fix dashboard data processing

mlco2 / codecarbon

Track emissions from Compute and recommend ways to reduce their impact on the environment.

https://mlco2.github.io/codecarbon

MIT License

1.12k stars 173 forks source link

Closed EdAbati closed 1 year ago

EdAbati commented 1 year ago

Fixes #423

Solutions:

Fixed pandas.DataFrame.to_dict("records")
Updated column names to reflect data in https://github.com/mlco2/codecarbon/blob/master/codecarbon/data/private_infra/global_energy_mix.json
Added unit tests to catch these tyupe of issues in the future
Extracted some manipulation in get_country_choropleth_data to make it easier to test

Please let me know if this is something you are interested in, and if I can improve this PR in any way :)

benoit-cty commented 1 year ago

Thanks, while testing your PR, I find other bugs :

per_capita_electricity is not in TWh, it is in Wh
per_capita_electricity is used in total_TWh computation, so it get it completely wrong, so the percentage was wrong in the end.
total_TWh also use *_Twh, but there are already other aggregates.
Errors in emissions.csv : azure and aws do not provide carbon intensity of their cloud.

I've pushed all these fixes, could you test it on your side ?

benoit-cty commented 1 year ago

Let me know what do you think. I am happy to make the fixes in the PR or in separate PRs, depends of what you prefer.

You could do everything in this PR. Thanks a lot !

EdAbati commented 1 year ago

Hi @benoit-cty , sorry for the delay! This should be ready now. Let me know if there is somenthing else I should change :)