mlco2 / codecarbon

Track emissions from Compute and recommend ways to reduce their impact on the environment.
https://mlco2.github.io/codecarbon
MIT License
1k stars 157 forks source link

Store data in smaller steps with reseting emissions data #597

Open headscott opened 2 days ago

headscott commented 2 days ago

Description

I found some issues from last year, where some people asked for saving emissions of loop steps seperatly without using checkpoints like flush or making new instances of the Emission Tracker. I can't find a way to get the emissions of each step still. I am not sure, if I just don't understand how I can do it.

What I Did

Right now I achieve this by creating a new instance of the tracker in each loop step like that:

def run_method():
    for i in range(1, 5):
        tracker = EmissionsTracker()
        tracker.start()
        # do stuff ...
        emissions: float = tracker.stop()

So it kind of works, but it always prints this:

[codecarbon INFO @ 15:02:39] [setup] RAM Tracking...
[codecarbon INFO @ 15:02:39] [setup] GPU Tracking...
....
....

And if I do this:

def run_method():
    tracker = EmissionsTracker()
    for i in range(1, 5):
        tracker.start()
        # do stuff ...
        emissions: float = tracker.stop()

It always prints this:

[codecarbon WARNING @ 15:04:02] Tracker already stopped !

and add the "kWh of electricity used since the beginning." value and Energy consumed for ... values up after each step