How to recover intermediate Code Carbon measurements?

[Lucas-Otavio]

• CodeCarbon version: 2.2.2
• Python version: 3.10.12
• Operating System: Ubuntu 22.04.3

Description

I would like to be able to plot the energy consumption of a project I'm developing in real-time. My idea was to use the consumption logs registered every 15 seconds or so, output it in a .log file, and read the lines with the information I want.

What I Did

I tried using OutputLogger to register it, but it only stored the final consumption values, just like emissions.csv does.

from codecarbon import EmissionsTracker
from codecarbon.output import LoggerOutput

import logging

# Create a dedicated logger (log name can be the CodeCarbon project name for example)
log_name = args.project_name
_logger = logging.getLogger(log_name)

# Add a handler, see Python logging for various handlers (here a local file named after log_name)
_channel = logging.FileHandler(f'{log_name}.log')
_logger.addHandler(_channel)

# Set logging level from DEBUG to CRITICAL (typically INFO)
# This level can be used in the logging process to filter emissions messages
_logger.setLevel(logging.DEBUG)

# Create a CodeCarbon LoggerOutput with the logger, specifying the logging level to be used for emissions data messages
my_logger = LoggerOutput(_logger, logging.DEBUG)

tracker = EmissionsTracker(project_name=args.project_name, 
                                   <other parameters>,
                                   save_to_logger=True,
                                   logging_logger=my_logger)

# setup

tracker.start()
# training call
tracker.stop()

Is it the expected behaviour? Is there an implemented way to retrieve these values in real time?

[benoit-cty]

Hello, For "real-time" we only support Prometheus or the CodeCarbon API. Could that fit your needs ?

[inimaz]

A bit more complex but feasible, one can create a custom output_handler and give it as input for the emissions tracker.

tracker = EmissionsTracker(...,
                           output_handlers=[ my_custom_output_handler])

This output_handler should be an implementation of https://github.com/mlco2/codecarbon/blob/master/codecarbon/output_methods/base_output.py.

See an example here https://github.com/mlco2/codecarbon/blob/master/codecarbon/output_methods/logger.py#L10

[Lucas-Otavio]

Thanks for the answers, I ended up doing a local copy of the project and changing Code Carbon's general logger, which worked for me. I will explore these options if I need to update my library version or use it in another project. Thanks!