search

lfwa / carbontracker

Track and predict the energy consumption and carbon footprint of training deep learning models.
MIT License
352 stars 26 forks source link

Log parser error due to zero power measurements #65

Open raghavian opened 4 months ago

raghavian commented 4 months ago

Sometimes the power_usage entries and duration arrays do not match in size causing an error in parsing at this line . Asserting for size should be introduced. Currently throws the following error.


File "/home/image/miniconda3/lib/python3.8/site-packages/carbontracker/parser.py", line 25, in parse_all_logs
    "components": parse_logs(log_dir, std, out),
  File "/home/image/miniconda3/lib/python3.8/site-packages/carbontracker/parser.py", line 54, in parse_logs
    energy_usages = (power_usages.T * durations).T
ValueError: operands could not be broadcast together with shapes (1,11) (10,)
'''