Capture measurements in CSV based on time intervals

[seriab]

• CodeCarbon version: 2.2.2
• Python version: 3.11.5
• Operating System: Linux-6.2.0-35-generic-x86_64-with-glibc2.35

Description

I have implemented a method that enables the capture and storage of consumption logs (CSV format) in parallel every second during the execution of my algorithm. Nevertheless, I would be interested in exploring whether there are more efficient alternatives from a technical or academic standpoint to accomplish this process. Below, I am providing the code of my current solution for review and suggestions.

What I Did

from codecarbon import EmissionsTracker
import threading

pthread=1
def fthread():
    while pthread:
        tracker.flush()
        time.sleep(1)

# Creates an instance of a thread and passes the function is executed.
eng_measure = threading.Thread(target=fthread)

# Training 
tracker = EmissionsTracker(project_name="Test",tracking_mode="machine")
tracker.start()
eng_measure.start()
try:
    # code
finally:
    pthread = 0
    eng_measure.join()
    emissions = tracker.stop()

[benoit-cty]

Hello, Doing it every second could be quite disk intensive, and useless as CodeCarbon only update measure every 15 seconds.

For a better solution we could update CodeCarbon to write to a csv file in the same place we call the API and Prometheus : https://github.com/mlco2/codecarbon/blob/master/codecarbon/emissions_tracker.py#L703

So you will just need to do: tracker = EmissionsTracker(project_name="Test",tracking_mode="machine", csv_run_name="emissions_test54.csv") And CodeCarbon will update the CSV every 8 x 15 seconds (api_call_interval * measure_power_secs).

What do you think of this ? Maybe CodeCarbon could generate a run_id to prevent the need of giving the file name.

[seriab]

Thank you very much for your response. The proposed update would be very useful. I will be looking forward to your reply.