search

sb-ai-lab / Eco2AI

eco2AI is a python library which accumulates statistics about power consumption and CO2 emission during running code.
Apache License 2.0
226 stars 18 forks source link

Function get_cpu_percent_linux gives ValueError when running for all CPU processes #5

Closed ioannismavromatis closed 1 year ago

ioannismavromatis commented 1 year ago

Hello,

When creating a CPU-based tracker on a Linux machine and trying to measure the power consumption for all CPU processed combined (cpu_processes="all"), I get the following ValueError:

Traceback (most recent call last):
  File "/home/synergia/.cache/pypoetry/virtualenvs/sustainable-ai-b93ea8Cf-py3.9/lib/python3.9/site-packages/apscheduler/executors/base.py", line 125, in run_job
    retval = job.func(*job.args, **job.kwargs)
  File "/home/synergia/.cache/pypoetry/virtualenvs/sustainable-ai-b93ea8Cf-py3.9/lib/python3.9/site-packages/eco2ai/emission_track.py", line 532, in _func_for_sched
    cpu_consumption = self._cpu.calculate_consumption()
  File "/home/synergia/.cache/pypoetry/virtualenvs/sustainable-ai-b93ea8Cf-py3.9/lib/python3.9/site-packages/eco2ai/tools/tools_cpu.py", line 151, in calculate_consumption
    consumption = self._tdp * self.get_cpu_percent() * self._cpu_num * time_period / FROM_WATTs_TO_kWATTh
  File "/home/synergia/.cache/pypoetry/virtualenvs/sustainable-ai-b93ea8Cf-py3.9/lib/python3.9/site-packages/eco2ai/tools/tools_cpu.py", line 132, in get_cpu_percent
    cpu_percent = os_dict[self._operating_system](self._cpu_processes)
  File "/home/synergia/.cache/pypoetry/virtualenvs/sustainable-ai-b93ea8Cf-py3.9/lib/python3.9/site-packages/eco2ai/tools/tools_cpu.py", line 534, in get_cpu_percent_linux
    index_cpu = strings.index(flag_string)
ValueError: '  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND' is not in list

This is due to how top prints on different terminals and terminal widths. In my setup top was printing one more empty space before PID.

I introduced a more generalised way of getting the results that matches top output using a substring instead of the entire line.

Try-except was also modified as top may produce results with processes marked as sleeping ('S' state). These processes don't consume CPU resources and, therefore, don't contribute to the overall power consumption.

vladimir-laz commented 1 year ago

Hello, ioannismavromatis!

Thank you for bringing the error in our project to our attention. We appreciate your efforts in trying to help us to improve. We have implemented our own solution to solve the issue, but we want to acknowledge your contribution. Your input is valuable to us, and we hope you will continue to support our project in the future. Thank you again for your help. Check it out, now cpu_percent calculating works correctly!

Best wishes, eco2ai team