Open jimbou opened 8 months ago
@russelltrow This is the repo for our project: https://github.com/TomasKopunec/comp0101-ief/ Do we need to reference it somewhere to submit?
Hi @jimbou here are the submission guidelines: https://github.com/Green-Software-Foundation/hack/wiki/Submit-your-project-for-judging
Please include your repo link under the "Artefacts" field.
This project includes the creation and integration of three unique plugins .The Carbon Advisor, Right-sizing, and Plotter plugins are intended to support organisations and software developers in making well-informed decisions that minimise energy use and lower carbon emissions without sacrificing functionality. Utilising the Carbon Aware SDK, the Carbon Advisor model determines the optimal carbon-efficient operating parameters—like time and place—for software programmes to function. The goal of the right-sizing model is to minimise wasted energy and capacity by optimising the use of cloud resources. Lastly, the Plotter model's user-friendly visualisations improve the clarity and accessibility of complicated environmental data.
Each of the 3 plugins try to add some missing functionality to the IEF Carbon aware advisor: The main lacking functionality of the IEF is that although it does provide a lot of measuring and monitoring capabilities, it does not offer suggestions on how to improve. With this plugin, we simplify the process of selecting the most sustainable energy sources by providing insight into carbon emissions data tailored to specific locations and time frames. Right-sizng: Again this plugin gives the user the opportunity to improve their carbon impact by finding more optimal cloud instances for their application without sacrificing performance. The benefits are also doubled since users can save on cloud costs as well. Plotter: The current IEF interface is based on yaml files. The format of the ompl file can be unfamiliar, and it can be hard to understand the impact and meaning of the data. Implementing a plugin capable of visualising the data makes the IF much more accessible and usable. This would result in more informed decision-making and a greater collective understanding among users.
Carbon Aware advisor: compares potential combinations of locations of cloud instances and time ranges of execution in order to determine the combination with the lowest carbon footprint for a given software system, using the carbon aware sdk webapi and with our own forecasting algorithm for future dates Right-sizing: Select a different cloud VM instance , in the same family for either azure or aws to maximaze CPU and RAM utilization and decrease price. Plotter: Offers the ability to create diagrams based on data in the IMPL file, data on csv format or data calculated in previous plugins in the piepline.
Best Plugin
Our plugins are meant to be used in collaboration with other plugins. For example the right sizing plugin can be integrated with the cloud-metadata. The latter brings the necessary information from the cloud , and the former right-sizes based on that data. The carbon advisor and plotter work really well with each other with the latter visualizing the data the former produces. In general the plotter plugin's true purpose is too visualize data produced by other plugins.
The plugins are self contained and work with both AZURE and AWS the 2 main cloud providers. They open the door for the impact engine not to just monitor and measure but to actually provide actionable suggestions , and thus really assisting in carbon impact reduction.
https://youtu.be/hbXzE4vl8ac There are multiple presentation videos and tutorials which can be found in our readme: https://github.com/TomasKopunec/comp0101-ief/tree/main
https://github.com/TomasKopunec/comp0101-ief/tree/main
We decided to start from scratch with a Node.js project instead of forking from the existing "if-unofficial-plugins.". To streamline our development process, we've implemented two execution modes: local mode and dev mode. Additionally, we've created shell scripts that automate compilation, file copying, plugin installation, and execution, accessible via shell commands or NPM scripts. This automation ensures that running the latest code requires no extra steps every time we change code. We also built 2 docker versions (mac, linux/windows) to run the plugins without any need for installation and avoid dependency issues. We used an agile way of development where we split into subteams that worked concurrently on the 3 plugins.
We were inspired by the problems we faced when using the IF. First of all the Ompl files were very diffuclt to read , understand and make assumptions from, thus we created the plotter. Also we felt that the IEF provided data but the user had to make its own conclusions and try to improve. So we though the carbon aware advisor and right-sizing models would be really helpful into showing the user how to improve.
The first challenge we faced was trying to predict future emissions since the carbon aware sdk had a 5 day limit. So we had to implement our own forecasting algorith the validity of which is difficult to test. But the main challenge was trying to keep up our models with the changes that were made to the IEF. At one point we had to rewrite the plugins from typescript to javascript since the IEF team decided that was the new standard way. So we had integration issues when the interface was changing. The way the IMPl files were written also kept changing so our examples would not work.
We are very proud to have created 3 functional models that will provide value imediateley to the current users of the impact engine. The docker version will also be very useful for anyone who wants to avoid the installation process and any potential compatibility issues.
The main thing we learnt is that it is not a good idea to implement plugins for a tool whose development is not yet finished. Having to keep up with the IF interface changes was as much work as the actual plugin development.
Plotter: more customisation abilities with new configuration options and graph types Right-sizing : more robust mechanism and new cloud providers Carbon aware advisor : more complex and modern forecastin option.
Prize category
Best Plugin
Overview
This project includes the creation and integration of three unique models inside the Impact Engine Framework (IF) to support green computing practices in response to the growing environmental concerns connected with the carbon footprint of information and communication technologies.
The Carbon Advisor, Right-sizing, and Plotter models are intended to support organisations and software developers in making well-informed decisions that minimise energy use and lower carbon emissions without sacrificing functionality. Utilising the Carbon Aware SDK, the Carbon Advisor model determines the optimal carbon-efficient operating parameters—like time and place—for software programmes to function. The goal of the right-sizing model is to minimise wasted energy and capacity by optimising the use of cloud resources. Lastly, the Plotter model's user-friendly visualisations improve the clarity and accessibility of complicated environmental data. Our extensive testing, which includes usability, performance, and unit evaluations, shows how well the models work to encourage software developers to embrace environmentally responsible practices. Future work will focus on refining these models and exploring additional avenues for reducing the tech industry's environmental footprint.
Carbon-aware-advisor The CarbonAwareAdvisor model is designed to provide carbon emission data based on specified locations and timeframes. It interacts with the Carbon Aware SDK to fetch the most carbon-efficient options for given parameters.
Right-Sizing The right-sizing model for Impact Engine Framework is designed to identify cloud computing instances that better align with the performance and capacity requirements of the customer's workload, with the goal of achieving the highest possible cpu usage , ram usage while minimising the cost and maintaining performance . It takes input in yaml format, ultimately providing more optimal instances based on the current cloud instance type, cloud vendor, current CPU utilization, target CPU utilization, and RAM requirements. Currently, this model primarily targets virtual machines of Azure and AWS.
Plotter The Plotter model created for the Impact Engine Framework is designed to visualize data through various types of graphs such as bar, line, and scatter plots. It takes input in YAML format or csv format , defining the x and y values along with additional parameters to customize the plots. This model is typically used in a pipeline following data-enrichment models like carbon-advisor, which populates the plotted-points parameter required by Plotter. If the user prefers he can specify the plotted-points parameter himself in the Impl file but the main value of the model is its ability to visualize the data provided by other models of the Impact Engine Framework. The user can also specify a csv file to read the data to plot from.
Questions to be answered
No response
Have you got a project team yet?
Yes and we aren't recruiting
Project team
@jimbou @GrayNekoBean @mosalehio @wangyiyuelva @pazmiller @TomasKopunec @JasonLuuk @TelmaGudmunds
Terms of Participation
Summary : This project includes the creation and integration of three unique plugins .The Carbon Advisor, Right-sizing, and Plotter plugins are intended to support organisations and software developers in making well-informed decisions that minimise energy use and lower carbon emissions without sacrificing functionality. Utilising the Carbon Aware SDK, the Carbon Advisor model determines the optimal carbon-efficient operating parameters—like time and place—for software programmes to function. The goal of the right-sizing model is to minimise wasted energy and capacity by optimising the use of cloud resources. Lastly, the Plotter model's user-friendly visualisations improve the clarity and accessibility of complicated environmental data.
Problem Each of the 3 plugins try to add some missing functionality to the IEF Carbon aware advisor: The main lacking functionality of the IEF is that although it does provide a lot of measuring and monitoring capabilities, it does not offer suggestions on how to improve. With this plugin, we simplify the process of selecting the most sustainable energy sources by providing insight into carbon emissions data tailored to specific locations and time frames. Right-sizng: Again this plugin gives the user the opportunity to improve their carbon impact by finding more optimal cloud instances for their application without sacrificing performance. The benefits are also doubled since users can save on cloud costs as well. Plotter: The current IEF interface is based on yaml files. The format of the ompl file can be unfamiliar, and it can be hard to understand the impact and meaning of the data. Implementing a plugin capable of visualising the data makes the IF much more accessible and usable. This would result in more informed decision-making and a greater collective understanding among users.
Application Carbon Aware advisor: compares potential combinations of locations of cloud instances and time ranges of execution in order to determine the combination with the lowest carbon footprint for a given software system, using the carbon aware sdk webapi and with our own forecasting algorithm for future dates Right-sizing: Select a different cloud VM instance , in the same family for either azure or aws to maximaze CPU and RAM utilization and decrease price. Plotter: Offers the ability to create diagrams based on data in the IMPL file, data on csv format or data calculated in previous plugins in the piepline.
Prize category Best Plugin
Judging criteria Our plugins are meant to be used in collaboration with other plugins. For example the right sizing plugin can be integrated with the cloud-metadata. The latter brings the necessary information from the cloud , and the former right-sizes based on that data. The carbon advisor and plotter work really well with each other with the latter visualizing the data the former produces. In general the plotter plugin's true purpose is too visualize data produced by other plugins.
The plugins are self contained and work with both AZURE and AWS the 2 main cloud providers. They open the door for the impact engine not to just monitor and measure but to actually provide actionable suggestions , and thus really assisting in carbon impact reduction.
Video https://youtu.be/hbXzE4vl8ac There are multiple presentation videos and tutorials which can be found in our readme: https://github.com/TomasKopunec/comp0101-ief/tree/main
Artefacts https://github.com/TomasKopunec/comp0101-ief/tree/main
Process We decided to start from scratch with a Node.js project instead of forking from the existing "if-unofficial-plugins.". To streamline our development process, we've implemented two execution modes: local mode and dev mode. Additionally, we've created shell scripts that automate compilation, file copying, plugin installation, and execution, accessible via shell commands or NPM scripts. This automation ensures that running the latest code requires no extra steps every time we change code. We also built 2 docker versions (mac, linux/windows) to run the plugins without any need for installation and avoid dependency issues. We used an agile way of development where we split into subteams that worked concurrently on the 3 plugins.
Inspiration We were inspired by the problems we faced when using the IF. First of all the Ompl files were very diffuclt to read , understand and make assumptions from, thus we created the plotter. Also we felt that the IEF provided data but the user had to make its own conclusions and try to improve. So we though the carbon aware advisor and right-sizing models would be really helpful into showing the user how to improve.
Challenges The first challenge we faced was trying to predict future emissions since the carbon aware sdk had a 5 day limit. So we had to implement our own forecasting algorith the validity of which is difficult to test. But the main challenge was trying to keep up our models with the changes that were made to the IEF. At one point we had to rewrite the plugins from typescript to javascript since the IEF team decided that was the new standard way. So we had integration issues when the interface was changing. The way the IMPl files were written also kept changing so our examples would not work.
Accomplishments We are very proud to have created 3 functional models that will provide value imediateley to the current users of the impact engine. The docker version will also be very useful for anyone who wants to avoid the installation process and any potential compatibility issues.
Learnings The main thing we learnt is that it is not a good idea to implement plugins for a tool whose development is not yet finished. Having to keep up with the IF interface changes was as much work as the actual plugin development.
What's next? Plotter: more customisation abilities with new configuration options and graph types Right-sizing : more robust mechanism and new cloud providers Carbon aware advisor : more complex and modern forecastin option.****