search

green-coding-solutions / eco-ci-energy-estimation

Eco CI Energy estimation for Github Actions Runner VMs
MIT License
48 stars 10 forks source link

Pre calculated Machine energy profiles #76

Closed ArneTR closed 2 weeks ago

github-actions[bot] commented 4 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run|28.5882|136.957|3.70154|70| |Sleep 3s|2.9|3.53779|1.7689|24| |ls -alhR /usr/lib|31.7416|128.113|4.00352|33| |Sleep 3s|1.17|5.30669|1.7689|3| 📈 Energy graph: ```bash 4.40 ┤ ╭╮ 4.14 ┤ ╭╮ ╭╮ ││ ╭──╮ 3.87 ┤ │╰───╯╰────────╯╰────────────╯ │ 3.61 ┤ │ │ 3.35 ┤ │ │ 3.08 ┤ │ │ 2.82 ┤ │ │ 2.56 ┤ │ │ 2.29 ┤ │ │ 2.03 ┤ │ │ 1.77 ┼─╯ ╰── Watts over time ``` 🌳 CO2 Data: City: Boydton, Lat: 36.677696, Lon: -78.37471 CO₂ from energy is: .05930238100000000000 CO₂ from manufacturing (embodied carbon) is: .01997196656202434362 Carbon Intensity for this location: 433 gCO₂eq/kWh SCI: 0.079274 gCO₂eq / pipeline run emitted
github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run|12.3439|84.0461|2.15|39| |Sleep 3s|4.06857||2.65|7| |ls -alhR /usr/lib|20.2364||3.05|16| |Sleep 3s|2.36857||2.06|8| 🌳 CO2 Data: City: Phoenix, Lat: 33.4475, Lon: -112.0866 CO₂ from energy is: .02857564000000000000 CO₂ from manufacturing (embodied carbon) is: .01112723851312768406 Carbon Intensity for this location: 340 gCO₂eq/kWh SCI: 0.039703 gCO₂eq / pipeline run emitted
github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run|11.0328|81.1451|2.08|39| |Sleep 3s|3.87333|15.7674|2.25|7| |ls -alhR /usr/lib|19.5436|48.9939|3.26|15| |Sleep 3s|2.39143|16.3838|2.04|8| 🌳 CO2 Data: City: Phoenix, Lat: 33.4475, Lon: -112.0866 CO₂ from energy is: .02929334500000000000 CO₂ from manufacturing (embodied carbon) is: .01112723851312768406 Carbon Intensity for this location: 361 gCO₂eq/kWh SCI: 0.040421 gCO₂eq / pipeline run emitted
github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run|12.76|73.4107|2495.96|34| |Sleep 3s|4.61286|19.1263|2.73|7| |ls -alhR /usr/lib|24.167|37.2294|3.38|11| |Sleep 3s|2.6|17.055|2.13|8| 🌳 CO2 Data: City: Washington, Lat: 38.7095, Lon: -78.1539 CO₂ from energy is: 0.031713120 CO₂ from manufacturing (embodied carbon) is: 0.009700669 Carbon Intensity for this location: 432 gCO₂eq/kWh SCI: 0.041414 gCO₂eq / pipeline run emitted
github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run|10.9871|83.3495|3250.63|39| |Sleep 3s|3.62857|18.8432|2.69|7| |ls -alhR /usr/lib|19.3179|48.6729|3.24|15| |Sleep 3s|1.62429|15.8334|1.98|8| 🌳 CO2 Data: City: Chicago, Lat: 41.8874, Lon: -87.6318 CO₂ from energy is: 0.035673372 CO₂ from manufacturing (embodied carbon) is: 0.011127239 Carbon Intensity for this location: 428 gCO₂eq/kWh SCI: 0.046801 gCO₂eq / pipeline run emitted
github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run|11.2707|84.4203|3207.97|38| |Sleep 3s|4.43429|18.9963|2.71|7| |ls -alhR /usr/lib|18.5679|48.3005|3.22|15| |Sleep 3s|2.33429|17.1236|2.14|8| 🌳 CO2 Data: City: Washington, Lat: 38.7095, Lon: -78.1539 CO₂ from energy is: 0.036131760 CO₂ from manufacturing (embodied carbon) is: 0.010841925 Carbon Intensity for this location: 428 gCO₂eq/kWh SCI: 0.046974 gCO₂eq / pipeline run emitted
github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run|10.6348|80.5664|3142.09|39| |Sleep 3s|2.81857|16.9856|2.43|7| |ls -alhR /usr/lib|19.1507|47.5853|3.17|15| |Sleep 3s|1.55|15.9955|2.00|8| 🌳 CO2 Data: City: Chicago, Lat: 41.8874, Lon: -87.6318 CO₂ from energy is: 0.034482248 CO₂ from manufacturing (embodied carbon) is: 0.011127239 Carbon Intensity for this location: 428 gCO₂eq/kWh SCI: 0.045609 gCO₂eq / pipeline run emitted
github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run|4.97043|51.0658|1583.04|31| |Sleep 3s|2.86167|14.319|2.05|7| |ls -alhR /usr/lib|10.9112|22.4127|2.80|8| |Sleep 3s|0.841429|14.3342|1.79|8| 🌳 CO2 Data: City: Chicago, Lat: 41.8874, Lon: -87.6318 CO₂ from energy is: 0.021855820 CO₂ from manufacturing (embodied carbon) is: 0.008844728 Carbon Intensity for this location: 428 gCO₂eq/kWh SCI: 0.030701 gCO₂eq / pipeline run emitted
github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run|10.4145|80.3311|3052.58|38| |Sleep 3s|2.29429|17.2344|2.46|7| |ls -alhR /usr/lib|18.8793|46.9488|3.13|15| |Sleep 3s|1.61571|16.1479|2.02|8| 🌳 CO2 Data: City: Boydton, Lat: 36.6534, Lon: -78.375 CO₂ from energy is: 0.034381668 CO₂ from manufacturing (embodied carbon) is: 0.010841925 Carbon Intensity for this location: 428 gCO₂eq/kWh SCI: 0.045224 gCO₂eq / pipeline run emitted
github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run|10.8774|83.472|3255.41|39| |Sleep 3s|4.52|19.0836|2.39|8| |ls -alhR /usr/lib|18.3564|48.1859|3.21|15| |Sleep 3s|1.92143|16.2025|2.03|8| 🌳 CO2 Data: City: Boydton, Lat: 36.6534, Lon: -78.375 CO₂ from energy is: 0.035726016 CO₂ from manufacturing (embodied carbon) is: 0.011127239 Carbon Intensity for this location: 428 gCO₂eq/kWh SCI: 0.046853 gCO₂eq / pipeline run emitted
github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run|5.31167|47.6964|1.54|31| |Sleep 3s|3.83|15.2323|2.18|7| |ls -alhR /usr/lib|7.17143|17.5336|2.19|8| |Sleep 3s|1.09429|14.9305|1.87|8| 🌳 CO2 Data: City: Chicago, Lat: 41.8874, Lon: -87.6318 CO₂ from energy is: 0.020747760 CO₂ from manufacturing (embodied carbon) is: 0.008844728 Carbon Intensity for this location: 435 gCO₂eq/kWh SCI: 0.029592 gCO₂eq / pipeline run emitted
github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run|10.6984|81.5383|2.09|39| |Sleep 3s|3.81143|18.0587|2.26|8| |ls -alhR /usr/lib|18.88|48.2922|3.22|15| |Sleep 3s|1.20143|15.1874|1.90|8| 🌳 CO2 Data: City: Chicago, Lat: 41.8874, Lon: -87.6318 CO₂ from energy is: 0.035469030 CO₂ from manufacturing (embodied carbon) is: 0.011127239 Carbon Intensity for this location: 435 gCO₂eq/kWh SCI: 0.046596 gCO₂eq / pipeline run emitted
github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run|10.7858|79.7241|2.10|38| |Sleep 3s|2.93571|17.6298|2.52|7| |ls -alhR /usr/lib|18.6343|46.9706|3.13|15| |Sleep 3s|1.18143|15.1236|1.89|8| 🌳 CO2 Data: City: Phoenix, Lat: 33.4475, Lon: -112.0866 CO₂ from energy is: 0.025511680 CO₂ from manufacturing (embodied carbon) is: 0.010841925 Carbon Intensity for this location: 320 gCO₂eq/kWh SCI: 0.036354 gCO₂eq / pipeline run emitted
ArneTR commented 2 weeks ago

The overhead is now down from 23s to 5s

It is unclear to me why the step for a measurement takes 5s. In local testing the time is less than 200ms

Even when I look at the detail logs the time seems to be less than 1s.

Screenshot 2024-06-14 at 8 30 26 AM
github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run (incl. overhead)|5.51167|52.515|1.69|31| |Sleep 3s|3.17667|14.1241|2.02|7| |ls -alhR /usr/lib|12.1875|22.9073|2.86|8| |Sleep 3s|1.32714|15.4836|1.94|8| 🌳 CO2 Data: City: Chicago, Lat: 41.8874, Lon: -87.6318 CO₂ from energy is: 0.022318875 CO₂ from manufacturing (embodied carbon) is: 0.008844728 Carbon Intensity for this location: 425 gCO₂eq/kWh SCI: 0.031164 gCO₂eq / pipeline run emitted
github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run (incl. overhead)|12.1103|85.3859|2.19|39| |Sleep 3s|5.62143|19.762|2.82|7| |ls -alhR /usr/lib|20.0214|49.3388|3.08|16| |Sleep 3s|2.85|16.2851|2.04|8| 🌳 CO2 Data: City: Boydton, Lat: 36.6534, Lon: -78.375 CO₂ from energy is: 0.036288625 CO₂ from manufacturing (embodied carbon) is: 0.011127239 Carbon Intensity for this location: 425 gCO₂eq/kWh SCI: 0.047416 gCO₂eq / pipeline run emitted
github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run (incl. overhead)|12.469|82.6879|2.12|39| |Sleep 3s|5.69|18.1724|2.60|7| |ls -alhR /usr/lib|20.3871|48.8222|3.25|15| |Sleep 3s|4.17571|15.6933|1.74|9| 🌳 CO2 Data: City: Chicago, Lat: 41.8874, Lon: -87.6318 CO₂ from energy is: 0.035141975 CO₂ from manufacturing (embodied carbon) is: 0.011127239 Carbon Intensity for this location: 425 gCO₂eq/kWh SCI: 0.046269 gCO₂eq / pipeline run emitted
ArneTR commented 2 weeks ago

I found the culprit. Aparently sending a process to the background makes GitHub steps hang for a while of the input and output streams are not handled.

This code part fixes it: https://github.com/green-coding-solutions/eco-ci-energy-estimation/commit/4425e1bd2371ac042c38f61f2b784719919d4a15

Brought it now also to this branch

github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run (incl. overhead)|9.12625|19.338|1.93|10| |Sleep 3s|6.565|5.64411|2.82|2| |ls -alhR /usr/lib|24.865|7.06576|3.53|2| |Sleep 3s|1.32|6.62815|2.21|3| 🌳 CO2 Data: City: San Jose, Lat: 37.1835, Lon: -121.7714 CO₂ from energy is: 0.001295646 CO₂ from manufacturing (embodied carbon) is: 0.002853138 Carbon Intensity for this location: 67 gCO₂eq/kWh SCI: 0.004149 gCO₂eq / pipeline run emitted
github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run (incl. overhead)|21.3288|53.699|2.98|18| |Sleep 3s|6.715|6.574|3.29|2| |ls -alhR /usr/lib|33.7367|39.4299|3.94|10| |Sleep 3s|2.37|7.69506|2.57|3| 🌳 CO2 Data: City: Boydton, Lat: 36.6534, Lon: -78.375 CO₂ from energy is: 0.023573861 CO₂ from manufacturing (embodied carbon) is: 0.005135649 Carbon Intensity for this location: 439 gCO₂eq/kWh SCI: 0.028710 gCO₂eq / pipeline run emitted
github-actions[bot] commented 2 weeks ago
Old Energy Estimation Eco-CI Output: |Label|🖥 avg. CPU utilization [%]|🔋 Total Energy [Joules]|🔌 avg. Power [Watts]|Duration [Seconds]| |---|---|---|---|---| |Total Run (incl. overhead)|19.3417|52.2195|2.75|19| |Sleep 3s|8.21|6.8245|3.41|2| |ls -alhR /usr/lib|28.8633|37.7405|3.77|10| |Sleep 3s|2.44333|7.65455|1.91|4| 🌳 CO2 Data: City: Phoenix, Lat: 33.4475, Lon: -112.0866 CO₂ from energy is: 0.015665700 CO₂ from manufacturing (embodied carbon) is: 0.005420962 Carbon Intensity for this location: 300 gCO₂eq/kWh SCI: 0.021087 gCO₂eq / pipeline run emitted
github-actions[bot] commented 2 weeks ago
Eco-CI Output: Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run (incl. overhead) 14.757 23.0745 2.10 11
Sleep 3s 6.445 5.65728 2.83 2
ls -alhR /usr/lib 26.5233 11.2577 2.81 4
Sleep 3s 0.856667 6.15952 2.05 3

🌳 CO2 Data: City: Washington, Lat: 38.7095, Lon: -78.1539 CO₂ from energy is: 0.010129486 CO₂ from manufacturing (embodied carbon) is: 0.003138452 Carbon Intensity for this location: 439 gCO₂eq/kWh SCI: 0.013268 gCO₂eq / pipeline run emitted

github-actions[bot] commented 2 weeks ago
Eco-CI Output: Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run (incl. overhead) 23.6765 56.5316 2.98 19
Sleep 3s 10.005 7.25165 3.63 2
ls -alhR /usr/lib 36.5856 41.2396 4.12 10
Sleep 3s 4.68 8.04038 2.68 3

🌳 CO2 Data: City: San Jose, Lat: 37.1835, Lon: -121.7714 CO₂ from energy is: 0.003787577 CO₂ from manufacturing (embodied carbon) is: 0.005420962 Carbon Intensity for this location: 67 gCO₂eq/kWh SCI: 0.009209 gCO₂eq / pipeline run emitted

github-actions[bot] commented 2 weeks ago
Eco-CI Output: Label 🖥 avg. CPU utilization [%] 🔋 Total Energy [Joules] 🔌 avg. Power [Watts] Duration [Seconds]
Total Run (incl. overhead) 23.9335 56.6945 3.15 18
Sleep 3s 13.445 7.49903 3.75 2
ls -alhR /usr/lib 35.6756 40.706 4.07 10
Sleep 3s 6.97667 8.48949 2.83 3

🌳 CO2 Data: City: San Jose, Lat: 37.1835, Lon: -121.7714 CO₂ from energy is: 0.003798498 CO₂ from manufacturing (embodied carbon) is: 0.005135649 Carbon Intensity for this location: 67 gCO₂eq/kWh SCI: 0.008934 gCO₂eq / pipeline run emitted