search

mlco2 / codecarbon

Track emissions from Compute and recommend ways to reduce their impact on the environment.
https://mlco2.github.io/codecarbon
MIT License
1.18k stars 178 forks source link

Divide by zero in `start_task()` #549

Closed AndrewLister-STFC closed 4 months ago

AndrewLister-STFC commented 6 months ago

Description

We are trying to use the task based emission tracker to give carbon benchmarks in fitting algorithms. To do this we run:

tracker.start_task()
runtimes = timeit.Timer(stmt=controller.execute).repeat(num_runs, 1)
emissions = tracker.stop_task().emissions / num_runs

This works on our linux machines but on windows it crashes due to a division by zero error in start_task().

What I Did

PS C:\Users\eel23943> python
Python 3.11.3 (tags/v3.11.3:f3909b8, Apr  4 2023, 23:49:59) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import codecarbon
>>> tracker = codecarbon.EmissionsTracker()
[codecarbon WARNING @ 18:06:38] Invalid gpu_ids format. Expected a string or a list of ints.
[codecarbon INFO @ 18:06:38] [setup] RAM Tracking...
[codecarbon INFO @ 18:06:38] [setup] GPU Tracking...
[codecarbon INFO @ 18:06:38] No GPU found.
[codecarbon INFO @ 18:06:38] [setup] CPU Tracking...
[codecarbon INFO @ 18:06:38] Tracking Intel CPU via Power Gadget
[codecarbon INFO @ 18:06:40] >>> Tracker's metadata:
[codecarbon INFO @ 18:06:40]   Platform system: Windows-10-10.0.19045-SP0
[codecarbon INFO @ 18:06:40]   Python version: 3.11.3
[codecarbon INFO @ 18:06:40]   CodeCarbon version: 2.4.1
[codecarbon INFO @ 18:06:40]   Available RAM : 31.441 GB
[codecarbon INFO @ 18:06:40]   CPU count: 16
[codecarbon INFO @ 18:06:40]   CPU model: 12th Gen Intel(R) Core(TM) i7-1270P
[codecarbon INFO @ 18:06:40]   GPU count: None
[codecarbon INFO @ 18:06:40]   GPU model: None
>>> tracker.start_task()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\eel23943\AppData\Local\Programs\Python\Python311\Lib\site-packages\codecarbon\emissions_tracker.py", line 479, in start_task
    _ = self._prepare_emissions_data(delta=True)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\eel23943\AppData\Local\Programs\Python\Python311\Lib\site-packages\codecarbon\emissions_tracker.py", line 611, in _prepare_emissions_data
    emissions_rate=emissions / duration.seconds,  # kg/s
                   ~~~~~~~~~~^~~~~~~~~~~~~~~~~~
ZeroDivisionError: float division by zero
AndrewLister-STFC commented 6 months ago

I've created a very hacky workaround for this in case anyone else sees this before there's a fix:

PS C:\Users\eel23943> python
Python 3.11.3 (tags/v3.11.3:f3909b8, Apr  4 2023, 23:49:59) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from codecarbon import EmissionsTracker
>>> t = EmissionsTracker()
[codecarbon WARNING @ 18:31:57] Invalid gpu_ids format. Expected a string or a list of ints.
[codecarbon INFO @ 18:31:57] [setup] RAM Tracking...
[codecarbon INFO @ 18:31:57] [setup] GPU Tracking...
[codecarbon INFO @ 18:31:57] No GPU found.
[codecarbon INFO @ 18:31:57] [setup] CPU Tracking...
[codecarbon INFO @ 18:31:57] Tracking Intel CPU via Power Gadget
[codecarbon INFO @ 18:31:58] >>> Tracker's metadata:
[codecarbon INFO @ 18:31:58]   Platform system: Windows-10-10.0.19045-SP0
[codecarbon INFO @ 18:31:58]   Python version: 3.11.3
[codecarbon INFO @ 18:31:58]   CodeCarbon version: 2.4.1
[codecarbon INFO @ 18:31:58]   Available RAM : 31.441 GB
[codecarbon INFO @ 18:31:58]   CPU count: 16
[codecarbon INFO @ 18:31:58]   CPU model: 12th Gen Intel(R) Core(TM) i7-1270P
[codecarbon INFO @ 18:31:58]   GPU count: None
[codecarbon INFO @ 18:31:58]   GPU model: None
>>> t.start_task()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\eel23943\AppData\Local\Programs\Python\Python311\Lib\site-packages\codecarbon\emissions_tracker.py", line 479, in start_task
    _ = self._prepare_emissions_data(delta=True)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\eel23943\AppData\Local\Programs\Python\Python311\Lib\site-packages\codecarbon\emissions_tracker.py", line 611, in _prepare_emissions_data
    emissions_rate=emissions / duration.seconds,  # kg/s
                   ~~~~~~~~~~^~~~~~~~~~~~~~~~~~
ZeroDivisionError: float division by zero
>>> from time import sleep
>>> hs = t._hardware[0].start
>>> def new_hs(*args, **kwargs):
...     sleep(1e-10)
...     return hs(*args, **kwargs)
...
>>> t._hardware[0].start = new_hs
>>> t.start_task()
benoit-cty commented 6 months ago

Thanks for reporting this !

LeahChercham commented 5 months ago

Hi @benoit-cty @AndrewLister-STFC

I proposed a pull request to solve this here: https://github.com/mlco2/codecarbon/pull/570

Basically I replaced all the time.time() by time.perf_counter() in the emissions_tracker.py file.

Hope this helps ! (Also, this is my first ever contribution to an open source project; please let me know if you have constructive criticism.) :)

benoit-cty commented 4 months ago

Thanks for your contribution to solve this issue !