search

mahendrapaipuri / jupyter-power-usage

Jupyter extension to display CPU and GPU power usage and equivalent CO2 emissions
Apache License 2.0
8 stars 0 forks source link

Work with CodeCarbon ? #6

Open benoit-cty opened 1 year ago

benoit-cty commented 1 year ago

Hello, Thanks for your tool ! I'm a member of CodeCarbon, do you thing we could work together to allow you tool use CodeCarbon as we already face the same problem you have ? But I don't know if a plugin coud be in Python as our package is in Pyhon ?

PS : CodeCarbon team is also mostly based in France and I discover your tool in Jean Zay Notebook.

mahendrapaipuri commented 1 year ago

Hello @benoit-cty, thanks for the issue.

Sure, I would be open to collaborate. What do you have in mind exactly? You would like to explore how to integrate CodeCarbon with this extension?

benoit-cty commented 1 year ago

For example we have :

mahendrapaipuri commented 1 year ago

Currently the server side extension exposes the RAPL power metrics and emissions are estimated in the browser. This is due to the fact that APIs to get emission factors are normally rate limited and so it is more logical to make those requests from user's browsers rather than servers.

I see a clear value on what you are proposing but I have following questions:

a worldwide database of carbon intensity

We can use this DB as a fallback for other countries. Currently we are using "real time" emission factor for France from RTE eCO2 mix data. Is it DB free and open source? Is it bundled along with CodeCarbon Python package?

Fallback to guess power if RAPL is not available

This would be indeed very useful as current extension works well only on bare-metals. We can still make it work in containers running on bare-metal but it is very complicated to make it work on VMs. Is it "guestimate" power exposed via CodeCarbon API? Something like:

from codecarbon import get_current_power_usage

print(get_current_power_usage())

Would an abstract API like above gives us current power consumption?

Handle SLURM allocation

This is unclear for me. Currently the extension can report metrics at process, user and system scope. So, if we want to get emissions from SLURM job, we can use user scope metrics inside that SLURM job.