Energy measurements for model inference

[LukasHedegaard]

Issue

In the current library version, the focus has been on tracking model training. A similarly important use-case for energy tracking is for model inference to gauge cost to our climate and wallets in deployment.

While one can "hack" this behaviour by specifying one epoch, which only includes inference, e.g.:

tracker = tracker.CarbonTracker(epochs=1, verbose=print_details)
tracker.epoch_start()
model(sample)  # inference here
tracker.epoch_end()

the energy usage is too low to be visible in the prints.

Proposed updates

1. Use higher precision for kWh print (or even give an option to print in Joules)
2. Create a separate mode, e.g. tracker.inference_start and tracker.inference_end, or even just tracker.start and tracker.end

[lfwa]

Hi Lukas,

I agree with your suggestions!

Regarding 1: You should be able to increase the decimal precision by setting the argument decimal_precision when you instantiate the Carbontracker class. There is currently an open issue where this only affects stdout and not the log files (see https://github.com/lfwa/carbontracker/issues/25). However, if the reported energy consumption is very low then the accuracy may not be very high. You should instead consider running inference for a longer period of time and then take an average.

If you would like to have the energy consumption printed in Joules instead the easiest option is likely to submit a pull request with the proposed changes (changes will likely be in tracker.py) or pull the repository, change it locally, and then install the package again from your local changes.

For 2: This is closely related to https://github.com/lfwa/carbontracker/issues/34 and I completely agree. However, further enhancements to carbontracker will likely have to be community-driven at this point in time due to all of us authors having other ongoing projects. If any critical issues appear then I will try to fix it as soon as possible.

[PedramBakh]

In Release v1.2.0 we've:

• Increased the default decimal precision to 12 to align with kWh and gCO2/kWh units, which are standards in the energy sector. This enhancement ensures consistent reporting without affecting existing functionality.
• Introduced the CarbonTracker CLI tool, making it simpler to integrate with model inference scripts or any other Python scripts you wish to monitor.

Given these updates, I'm closing this issue. If there are any further concerns or feedback related to this topic, please feel free to reopen or create a new ticket.