Open pierresegonne opened 1 month ago
hanopcan
commented
1 month ago Hi @pierresegonne - thank you so much for this thoughtful comment. It was always my hope that this work could be refined, questioned and built upon by a wider community so I am delighted by your engagement.
As I mentioned to you in a private DM recently, please give me a few days to read and digest everything you are saying here so I can figure out how to respond in a meaningful way. Will reply soon!
hanopcan
commented
3 weeks ago Hi @pierresegonne
I've had chance to read through and digest your comments in detail - again thank you for all the effort you put in to communicate those points so clearly. This has been a really interesting read.
I realise it's been nearly a year since myself and Ismael (@Leamsi9) produced this initial thinking and repo exploring what the issues we thought were were seeing in carbon-aware, and what a better position might be eg the initial concept of grid-aware. Since then we've been in numerous places discussing this with others, and no doubt our thinking has matured. Your comment and thoughts are a great prompt to step back and re-examine those initial conclusions.
In short, I feel that it's time to publish a next version and I'd love to incorporate your thoughts into that, giving you full credit of course Pierre.
I think your points make sense and I think I agree with your conclusions. I'm still working them through in detail and testing them out, which usually happens for me in a writing process (eg working them into a v2). I also really appreciate your revised hierarchy for impact (and also that you gave the initial three bullet points that name), this is brilliant:
+++ Certainty, verifiability of impact
Reduce total energy usage
Displace, consistently, compute to areas where low-carbon electricity is abundant, and where there are credible plans for making the grid even cleaner
Run compute on additional low-carbon electricity - i.e. pick the right provider for your compute
Time shift consumption to match low demand OR low carbon intensity, based on the location
--- More uncertain, hard to assert claims about emission reductThis revised hierarchy feels very central to what the write-up should focus on.
My planned next step is to create a new branch of this repo and bring in your points to expand on what's there. And even correct, remove a few parts too.
When I get to that would you like to be involved and review @pierresegonne? 😁
pierresegonne
commented
3 weeks ago I'm glad to read this @hanopcan !
Please reach out whenever your v2 is mature enough to be reviewed, and I'll do my best to set time aside to review :) Note that this is correlated but not central to my work at Electricity Maps and I might not be able to review immediately.
In any case, I'll be at GridIO in Paris in December :)
Hey there,
Thanks a lot for putting this work together! I came across that much needed starting point in the discussion around optimising compute for GHG emissions listening to GreenIO's podcast.
First off, I'm really impressed that you both had the courage to question the status-quo and some of the claims of the large players in that space regarding optimising compute to minimise carbon emissions. I work for Electricity Maps, where our mission is to enable a data driven decarbonisation of electricity. I specifically work in the team generating grid forecasts, where we aim to develop the most meaningful forecasts to enable anyone to take part in that decarbonisation effort. That ranges from providing forecasts that can be embedded in home appliances, utilised for EV smart charging, all the way to providing the forecasts used by Google for load-shifting in their data centers. Generally, we want to be part of that discussion as our forecasts can be one of the cornerstones on which carbon-aware computing is sometimes based. Eventually, what we want to ensure is that the forecasts we develop are indeed useful for making a positive difference and not only as a "feel good" mechanism.
With that in mind, I'd like to share thoughts and feedback on the work presented here, based on how I've have been thinking about this problem. This does not necessarily reflect 1:1 how Electricity Maps as a whole as been thinking about the problem, and more about I've personally tried to become smarter around that topic.
A hierarchy for impact
Generally, I really like the idea of presenting a hierarchy of action one could take to reduce overall emissions from computing. From what I could infer from the podcast and the documents here, it seems like the way you've been thinking about it is:
For 1., there is no debate around that. The most certain and efficient way to reduce emissions is to reduce energy consumption.
For the two other points, I think there is a discussion to be had. I think it comes down to a question of certainty in the impact of one's action and verifiability.
Run compute when demand is low, with curtailed electricity if possible
-> The problem I see in practice with moving compute to when demand is low is that the same problem you identified applies as when one shifts consumption to match the lowest carbon intensity. I.e the impact of doing that on the grid is not necessarily super clear.
About the grid As you both highlighted, understanding how the grid reacts to a load increase is really complex. The entrenched idea that production = consumption at all times is a simplifying trope that obscures how the grid actually operates. When someone adds a load to the grid, the reaction will be conditioned by the size of that load in regards to the grid. For large synchronous grids like the European one, the addition of up to e.g 50 kW of demand (that's ~a thousand computers suddenly starting) will not be met by any reaction of the grid. There might be issues at the distribution level, but it will not trigger any additional power generation. This is due to the sheer size of the grid, this additional demand is just a drop in the ocean, meaning that the overall frequency response of the grid will be insignificant. I'm still looking into understanding better where the limit of that property lies. This report by ENTSO-E on the integration of EVs can provide a good understanding of how the grid reacts to these flexible loads. They explicitly state
Meaning that the order of magnitude of tens of kilowatts is too small to make any difference in terms of the frequency of the grid (ancillary services are used to regulate the frequency of the grid within small time frames, seconds to minutes).
On the other side of the spectrum, there are instances where huge loads differential are really easy to observe, and that triggers a long chain of consequences (Ex: the sudden trip in the interconnect between Great Britain and Norway, which was operating at 1.4 GW, which triggered the activation of ~80% of all reserve capacity to stabilise the grid). To me, the discussion around shifting demand to whatever time should therefore really include a question about the size of that load, and should be highlighted in that discussion.
About low demand as a proxy I think that as you argue for, targeting low demands periods is generally a pretty good heuristic in the short term for someone who has little access to accurate and granular electricity data, and is just starting thinking about this topic. I nevertheless think that this should come with a quite large asterisk. Here are two examples, completely cherry-picked, browsing our app, that can serve as counter examples:
For Spain, I can see that the period of low demand just before the morning is seeing a ramp-up of gas production (grey). Why would one then shift its consumption to a period where low-carbon sources are already operating at maximum capacity and gas is at time picking up to meet the increasing demand? (not necessarily from a marginal point of view - more from what we see, gas is being ramped up in the hours following the lowest load). In the long run that would create a deleterous incentive - it would actually be encouraging gas power plants to operate more hours throughout the day!
For Bosnia-Herzegovine, it seems like they are matching demand with hydro throughout the day but rely on much dirtier imports (yellow, vs-hydro) during the low demand periods to match whatever comes on the baseload. In that case it seems counterproductive to encourage demand that would be met with these imports instead of the local hydro.
So overall, recommending targeting periods of low demand with shifted consumption as a go-to strategy should be taken with a grain of salt (as much as targeting highest shares of renewable!), as the validity of that approach will be dependent on where/when this optimisation applies!
The other elephant in the room: short-term vs long-term thinking
It seems like the conclusions you draw, especially for the identified problems with carbon-aware software are not so specific about the time horizon of the consequences discussed.
When I try to understand the effect of displacing in time or space a given load, that parameter plays a decisive role.
The train analogy: In the podcast (I think it's somewhere in this repo as well, but I could not find it at this time, or maybe is it in Ismael' post on Hackernoon?), is presented an analogy to illustrate the effect of time shifting consumption based on the carbon intensity of electricity. To rephrase, the idea is that for each hour of the day runs a train, with a set number of passengers per journey. That train is made up of two different classes of wagons, some are labeled as "low-carbon" while the rest are dirty. Load shifting would then amount to being able to select what train to travel on, and thus informs our likelihood of being able to sit down in a "low-carbon" wagon. I like this analogy to explain the short term effect of load-shifting. Indeed, when shifting a small load (see point above), we will not produce much of a positive difference. The train, with all its associated emissions, will still run no matter which one one takes!
I think where that analogy is lacking is considering that the make-up of these wagons is not something set in stone. The number of wagons and the make-up will evolve over time. Let's say there are two entities providing these trains, with one that has a serious plan for increasing the share of low-carbon wagons over the next 2 years. Which one would you choose to ride with?
If I recall correctly, an objection provided around this is that low-carbon sources have already "won"; they're cheaper to build and operate, faster to install etc. I can't find the reference, but a mention of the fact that some European nations were already ahead of their sustainable goals related to energy was also made. The goals mentioned here are sustainable development goals from the UN, and those reached by some European countries are for energy access, and have nothing to do with clean energy deployment! It is true that nowadays solar and wind are hugely competitive for new installed electricity capacity, but that does not mean that the problem is already solved! I would encourage you to have a look at this outlook by Ember. You can see that the ambition by 2030 for the US is to reach ~60% of renewable electricity generation, while for Japan it's only 38% and India ~42%, and the level of certainty around these countries reaching that goal can be quite low.
That means that going forward, not all locations are equals in terms of how much of the additional demand on the electricity grids will be met by low-carbon sources. I therefore find it quite premature to dismiss the idea of location shifting all together. Why not encourage compute to be located in places that already have access to loads of low-carbon electricity and that have ambitious and credible plans for the development of their grid based on low-carbon resources?
The long-term impact of shifting loads in time is not something easy to model. It's nevertheless possible to see that there is a problem that exists today where flexible compute could help. In certain locations, the renewable penetration is such that in times of high production, the total demand is not sufficient to consume all electricity produced. That results in negative prices (Germany is becoming synonym with that) or curtailment. A big part of the problem here is the grid configuration itself as you highlighted in the podcast (wind electricity produced in Scotland but that can't be transferred to Southern England, where consumption happens; or similarly in Germany), and that's going to take some time to solve. Moving computing resources close to where that extra production regularly happens and letting it match periods of high production is a clear cut solution. Even if it's impossible to find a data center close to wind turbines or solar panels that are being curtailed, encouraging consumption at times where there is a lot of low-carbon generation is a clear incentive for the grid to invest in upgrades to serve the locations that are willing to consume that extra electricity.
I would therefore argue that the whole section around problems with carbon aware software must be expanded to account for that part of the question. Dismissing completely the idea of time and location shifting based on a short-term vision obscures a part of the answer: creating the right incentives for using low-carbon electricity where it's available and where it's going to still be available in the future.
Run compute on additional low-carbon electricity
Thanks for bringing up the part around the additionality of the electricity used. That's a really important aspect to take into account when choosing e.g. a cloud provider. My only feedback here would be to encourage practitioners to make their due diligence when choosing compute providers. I'm not going to provide names nor point fingers (this article by the Financial Times is a good read with that regard), but not all providers are as transparent and as committed to supplying their operations with additional low carbon electricity.
A new hierarchy for impact
Based on the comments above, I would propose the following hierarchy for impact:
+++ Certainty, verifiability of impact
--- More uncertain, hard to assert claims about emission reductions
This feedback is quite a mouthful, but I hope it will help the conversation move forward. As you correctly pointed out, now is the time to make sure we do this right, and I would love to be a part of that.
I would love to discuss directly as there are, I'm sure, other topics where a fruitful discussion could happen :) Feel free to connect with me on LinkedIn