mlco2 / codecarbon

Track emissions from Compute and recommend ways to reduce their impact on the environment.
https://mlco2.github.io/codecarbon
MIT License
1k stars 157 forks source link

Adding and Choosing TDP values #532

Closed Lucas-Otavio closed 1 month ago

Lucas-Otavio commented 2 months ago

Description

I was trying to track the emissions of a code and received this message:

[codecarbon WARNING @ 14:50:23] No CPU tracking mode found. Falling back on CPU constant mode.
[codecarbon WARNING @ 14:50:24] We saw that you have a Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz but we don't know it. Please contact us.

What I Did and wish was done

I searched for the proper TDP and did a PR to add it to the CSV file, but I wanted to be able to track it with the right reference value without waiting for a new release. It would be great if you could add a parameter to the function, allowing us to choose the TDP.

Moreover, there is a 3 times larger file on the eco2ai project, which could be very useful to avoid repetitive PRs to add one single TDP. I think I could contribute by writing some code to compare the two, add the missing ones and find inconsistent values (if any).

benoit-cty commented 2 months ago

Hello, It would be great to automate this part, yes !

Another way is to look at this code : https://github.com/Breakend/experiment-impact-tracker/blob/master/experiment_impact_tracker/cpu/data/scrape_tdp_values.py See https://github.com/mlco2/codecarbon/issues/400

You could already specify your TDP using the parameter default_cpu_power.

But I realize that it has not been documented in https://mlco2.github.io/codecarbon/parameters.html , sorry.

Lucas-Otavio commented 2 months ago

Thanks for the clarifications.

I checked that code and it seems they used a Wikipedia page that doesn't exist anymore. Maybe it could be achievable using the Intel one and the pages on lists of AMD processors Series (such as this).

Lucas-Otavio commented 2 months ago

Hello,

I have begun doing the comparison I mentioned. Merging the 2 files, it reaches 3749 processors. Moreover, there are 174 with contrasting values. In 39 of them, the difference seems to be because eco2ai's value is in float, while codecarbon's is in int.

Taking a look at the processor specifications for some of the other incompatible TDPs, in some of them codecarbon has the right value; in others, eco2ai.

Below, I'm sending the table and incompatibility analysis. Hoping to make a pull request, I wanted to know how to deal with the incompatible values. Should I add them to the table with the previous value? Would it create a problem if I inserted float values? Thanks in advance for the feedback.

merged_table.csv incompability_analysis.txt => cpu_power is the file from code carbon and cpu_names the one from eco2ai

benoit-cty commented 2 months ago

Nice job ! In doubt you could keep CodeCarbon value.

You could try to keep float, I saw there is CPU with TDP around 1W so it make sense.

Lucas-Otavio commented 2 months ago

Okay, I will do the pull request using CodeCarbon's values and resend the incompability list. Maybe the issue should remain open or another one should be posted to ask for reviewing those values. I checked some using intel specifications and wikichip.org for AMD chips; and corrected the value when the right one was clear.

In the latter case, it seems Code Carbon's value is rather TDP up instead of TDP for AMD chips. I am not sure which one it should be, so left it as it is in those cases.

While using wikichip I found the following links that may be usuful for webscrapping in the future:

Also, since eco2ai uses an Apache license, I believe some notice should be added.

incompatibility_analysis.txt

benoit-cty commented 2 months ago

Thanks a lot !

SaboniAmine commented 1 month ago

Closing as PR have been merged.