mlco2 / codecarbon

Track emissions from Compute and recommend ways to reduce their impact on the environment.
https://mlco2.github.io/codecarbon
MIT License
1.15k stars 173 forks source link

Question about accuracy of carbon tracker when CPU is unknown and gpu_id not set #583

Closed headscott closed 3 months ago

headscott commented 4 months ago

Description

Hey, I am trying to use CodeCarbon to check if some AI models have other power consumptions then others. So I would like to have the exmissions to be tracked as good as possible for the exact process I am running. But I got some warnings:

[codecarbon WARNING @ 22:22:26] Invalid gpu_ids format. Expected a string or a list of ints. [codecarbon WARNING @ 22:22:26] No CPU tracking mode found. Falling back on CPU constant mode. [codecarbon WARNING @ 22:22:27] We saw that you have a AMD Ryzen 5 3500 6-Core Processor but we don't know it. Please contact us.

Will the results be good anyways, or should I definetly try to give an gpu_id and set any values for my CPU? What gpu_id should I give the tracker, how and why?

And does tracking_mode set to "process" work fine? Or should I track the whole system, but try not to run too many programs in the background?

What I Did

from codecarbon import EmissionsTracker
import foolbox as fb
import numpy as np

def run_attack():
    tracker = EmissionsTracker()
    emissions_list = []

   #loading my AI model and test data...

    for i in range(1, 5):
        tracker.start()
        #attack the model and print Robust accuracy
        emissions: float = tracker.stop()
        emissions_list.append(emissions)

    # Berechnung von Mittelwert und Standardabweichung
    mean_emissions = np.mean(emissions_list)
    std_emissions = np.std(emissions_list)

    print(f"Mean emissions: {mean_emissions} kgCO2eq")
    print(f"Standard deviation of emissions: {std_emissions} kgCO2eq")

# This will be run when I press the green button inside my IDE
if __name__ == '__main__':
    run_attack()
inimaz commented 4 months ago

Hi @headscott, thanks for using codecarbon!

There are 3 warnings in your logs:

[codecarbon WARNING @ 22:22:26] Invalid gpu_ids format. Expected a string or a list of ints.

This was a bug and it has been already fixed. We will do a new release soon, it will be included in that one.

[codecarbon WARNING @ 22:22:26] No CPU tracking mode found. Falling back on CPU constant mode.

This means that codecarbon has not found a way to track your CPU live. If you would like to do so, install Intel Power Gadget. Otherwise it will go into CPU constant mode. See here for more info about how codecarbon works.

[codecarbon WARNING @ 22:22:27] We saw that you have a AMD Ryzen 5 3500 6-Core Processor but we don't know it. Please contact us.

We keep a list of TDPs per processor here. Could you open a PR adding yours to that list?

inimaz commented 4 months ago

And does tracking_mode set to "process" work fine? Or should I track the whole system, but try not to run too many programs in the background?

At the moment the tracking_mode "process" works only for RAM measurements. There is a PR open for CPU #316, which is ongoing.

headscott commented 4 months ago

Could you open a PR adding yours to that list?

Do you mean a pull request?

headscott commented 4 months ago

I tried to download the Intel Power Gadget, but when running the pcm.exe it says:

Starting MSR service failed with error 577 ...

and with Bcdedit.exe -set TESTSIGNING ONafter reboot:

DEBUG: Setting Ctrl+C done.

Intel(r) Performance Counter Monitor ($Format:%ci ID=%h$)

Error: unsupported processor. Only Intel(R) processors are supported (Atom(R) and microarchitecture codename Nehalem/Nehalem-EP, Atom(tm), Westmere/Clarkdale, Sandy Bridge, Westmere-EP, Sandy Bridge-EP/Jaketown, Nehalem-EX, Westmere-EX, unknown, Centerton, Baytrail, Ivy Bridge, Haswell, Broadwell, Ivy Bridge-EP/EN/EX/Ivytown, Haswell-EP/EN/EX, Cherrytrail, Avoton, Skylake U/Y, Broadwell-EP/EX, Skylake-SP, Cascade Lake-SP, Broadwell-DE, Knights Landing, Apollo Lake, Skylake, Denverton, Icelake-SP, Gemini Lake, Icelake, Snowridge, Tiger Lake, Kabylake/Whiskey Lake, Sapphire Rapids-SP, Alder Lake, Kabylake, Comet Lake, Rocket Lake, Meteor Lake, Sierra Forest, Raptor Lake, Emerald Rapids-SP). Access to Intel(r) Performance Counter Monitor has denied (no MSR or PCI CFG space access).

benoit-cty commented 3 months ago

Closing in favor of https://github.com/mlco2/codecarbon/issues/457