mlco2 / codecarbon

Track emissions from Compute and recommend ways to reduce their impact on the environment.
https://mlco2.github.io/codecarbon
MIT License
1.02k stars 158 forks source link

required: clarity over what is recorded (varies by version? machine?) #363

Closed mkbane closed 1 year ago

mkbane commented 1 year ago

Description

Two different Linux machines, but running same Python3 code I get different behaviours.

On Debian10 (albeit a VM within vmware under Win10), I get 1 line of data plus the headings loud_provider,cloud_region,country_iso_code,country_name,cpu_count,cpu_energy,cpu_model,cpu_power,duration, emissions,emissions_rate,energy_consumed,gpu_count,gpu_energy,gpu_model,gpu_power,latitude,longitude, on_cloud,os,project_name,python_version,ram_energy,ram_power,ram_total_size,region,run_id,timestamp,tracking_mode

whereas on Linux-3.10.0-1160.36.2.el7.x86_64-x86_64-with-centos-7.9.2009-Core,3.6.8 I get 101 lines of data plus the headings imestamp,project_name,run_id,duration,emissions,emissions_rate,cpu_power,gpu_power,ram_power,cpu_energy,gpu_energy,ram_energy,energy_consumed,country_name,country_iso_code,region,cloud_provider,cloud_region,os,python_version,cpu_count,cpu_model,gpu_count,gpu_model,longitude,latitude,ram_total_size,tracking_mode,on_cloud

The Debian10 instance does give warnings: [codecarbon WARNING @ 22:22:09] No CPU tracking mode found. Falling back on CPU constant mode. [codecarbon WARNING @ 22:22:11] We saw that you have a Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz but we don't know it. Please contact us.

The Centos run does have access to RAPL counters.

I am very confused as to why the different behaviour. despite the different hardward (and slightly different Python versions IIRC)

Related(?) I see the Debian10 run also creates (leaves?) a number of emissions.csv_NNN.bak (for NNN=0,1,2,...) files

benoit-cty commented 1 year ago

Hello, Thanks for reporting this. When you say "101 lines of data" you mean in the CSV file for only one run ? That's not what is suppose to happen. The *.bak are created when we do not found the expected header in the csv file, so we made a backup before fixing it. It is supposed to happen only one time.

We have to look a the code to understand what happen.

benoit-cty commented 1 year ago

Closing because of lack of answer, feel free to reopen.