Closed robbieorvis closed 6 months ago
jrissman
commented
1 year ago You can try the version from commit 898166d, which didn't have the overcompliance compensation mechanism. It works quite cleanly in that commit, if you don't mind that it makes all hours approach their final compliance targets rather than letting some hours of excess compensate for a shortfall in other hours. I probably should have let you review 898166d before building anything on top of it.
jrissman
commented
1 year ago Brief summary here of recent work, which has focused on the RPS/CES policy:
Robbie's suggested approach using an optimization loop to find a CES price seems to be working well after some minor fixes, and I think we should keep it, deleting my earlier approach that involved looping through the hours of each timeslice and building to meet RPS in each hour, then trying to compensate for hours where we overcomply. Robbie's approach is more cost-optimized and also generates a CES credit price we can use as an input to the profitability-based capacity construction mechanism.
To reliably hit an RPS target, it is necessary to move all demand-shifting technologies (like grid batteries, pumped hydro, demand response programs, etc.) upstream of the capacity construction calculations, because they affect the amount of electricity demand in each hour, which affects the RPS target. With them coming later, they introduce noise that make precise RPS calculations impossible. Moving them upstream means the hours in which demand shifts occur will be based on current-year electricity demand and the potential output from surviving capacity, which is like it is today except excluding capacity built in the current modeled year. That should be fine because the vast majority of capacity in any given year is surviving, not newly built. Also, it makes more sense that electricity system operators would try to optimize their system as best they can using existing resources like demand response before committing to build new plants, rather than first committing to build plants, then using their existing resources to optimize around the plants they semi-blindly built. I haven't done this work yet. But when I disable the demand-shifting technologies, the RPS is met beautifully in every year (or exceeded in early years, when the RPS policy is weak), with no error like we were seeing before. It's a nice improvement and also facilitates debugging and understanding what the model is doing.
There is still a surge of construction in the year an RPS policy reaches 100% because it needs enough resources to comply in every year. There are really two approaches to address this:
I first moved diurnal grid batteries over so they would help meet the RPS in some hours by taking from hours where there is excess electricity. I got this working but discovered it didn't reduce the spike of construction in the final year at all, since there are some days (Summer Peak Day Three) where there were no hours with excess electricity, so the batteries had no opportunity to charge, so they could not discharge. The system therefore still needed to build the resources required to hit the RPS on Peak Summer Day Three, and the batteries made no difference.
After a discussion with Megan, we realized we needed seasonal storage, to shift from times of abundant RPS-qualifying electricity to times of shortfall. I tried shifting solely within the same season (i.e., from average summer days to peak summer days and average winter days to peak winter days), but this didn't work, as even the average days had many hours with significant undercompliance with the RPS in the final year, so they were not good candidates to source power from. We needed a more generalized and optimized approach to deciding where to take power from.
I completed a generalized implementation with both seasonal and diurnal storage capabilities in 8d7536d. With enough seasonal and diurnal storage, it is able to cut down on the amount of capacity built in the final year without affecting the ability to hit the RPS and without overcompliance in earlier years. It's the first time we've seen any way to do that, so it feels like a breakthrough. The one caveat to the seasonal and diurnal storage is that you need quite a bit of seasonal storage to make a difference on the amount of capacity that needs to be built to meet the RPS. It just takes a lot of storage to make up the shortfall, particularly because there is a shortfall in many hours of "average summer day" and to a lesser extent in the other "average" days, which are harder to make up because they each are weighted by many days, so you need to transfer a lot of energy to start reducing those shortfalls noticeably. Treating all this capacity as "grid batteries" is probably not accurate, so we might want to consider it just "seasonal storage" and make it a sum of all relevant storage types (like pumped hydro, etc.). Also, we should add multipliers for some of them based on the average number of times they cycle per year, to get the seasonal storage aggregate quantity that we can use to shift electricity around.
It still doesn't cause pre-building in earlier years, of course, so there still is a spike in the last year. It's just smaller than it was.
This bring us to the other mechanism for reducing last-year construction in a 100% RPS scenario: RPS foresight and pre-building. By definition, having RPS foresight and building some resources earlier will necessarily cause over-compliance in earlier years. This is an inescapable trade-off between overcompliance and using pre-building to reduce the last-year construction spike.
Still, this is a potentially important mechanism, and we always have had it in the EPS, so we should retain it, with a control lever to enable or disable it. Then, an individual user can decide if they prefer to use some pre-building with overcompliance of the RPS in earlier years to lessen the construction spike when the RPS reaches 100%, or if they prefer to follow the RPS narrowly and accept the spike at the end. I don't think this is a modeling error - I think these are the real choices a real-world utility would face as it goes past 96% clean energy and is thinking about how to get that last 4%.
Sufficient seasonal and diurnal storage can help, but the modeling shows it's not a magical panacea.
Once I get this all cleaned up and the demand-shifting step moved upstream of the capacity construction step, I'm going to consider work on the RPS policy done for the next release. It's working better than ever before, and I'm increasingly convinced the remaining trade-offs are real world challenges, not modeling artifacts.
jrissman
commented
1 year ago One potential way to simplify the handling of storage is to take in the following info about each storage technology:
We multiply capacity (MWh) by number of cycles per year (dimensionless) to get the total number of MWh the storage can absorb. It discharges the same amount, adjusted for the round-trip efficiency. We divide up the total amount discharged between helping to meet the RPS and helping to reduce net peak load based on the user-specified share noted in the last bullet above.
For the MWh discharged to help meet the RPS, we allocate them across all hours and timeslices proportionally according to need (i.e., the shortfall relative to the RPS value). For MWh discharged to help reduce net peak load, we similarly allocate it proportionately to the net load.
If restricting some storage mechanisms to diurnal operation is important and is worth the extra complexity, we can retain the new structure that does this, applying it to all diurnal modes (say, some grid batteries and demand response), while using the seasonal structure for other modes (some grid batteries, pumped hydro, etc.). We shouldn't need separate passes for each individual storage technology - we only need calculation passes for each of the two broad operation modes, diurnal and seasonal. This should be a big improvement over the structure that takes each demand shifting technologies in sequence and restricts most or all of them to diurnal operation.
mkmahajan
commented
1 year ago I just spent a bit of time looking at NREL's scenario viewer for their 100% clean electricity by 2035 report series. Interestingly, their main case allows DAC to offset fossil generation from the power sector, so they only really get to ~95% clean generation. It's therefore hard to evaluate whether they'd need a high build rate in the last few years to meet a 100% target.
But they do have a side case that excludes all CCS technologies. In that side case, you see a large jump in H2 power plant and solar capacity additions in 2034-2035 in order to get the last few percent of clean generation. Their battery storage additions, which I assume are 4-hour batteries, don't have that same type of spike in the last few years. This gives me some confidence in the fact that diurnal storage isn't the limiting factor, but that instead you need some amount of clean flexible generation like CT plants with CCS, hydrogen power plants, or other seasonal storage.
NREL's model runs are some of the most trustworthy out there, and they also have the same type of result we're finding. But I still like the idea of adding a toggle to enable foresight to spread out the construction of these resources over several years, which could allow us to show a range in outcomes
Here is a graph of capacity in their No CCS case, and the blue and orange bars closest to the x-axis show new-build and retrofit H2 CT plants, respectively. the brown bar at the top shows H2 electrolyzer capacity. you can see those three data series suddenly pop up in 2034 and 2035.
jrissman
commented
1 year ago This is a great finding. I think our model now is showing what NREL's model shows: batteries just aren't enough to really prevent the need to build noticeably more capacity when the RPS gets past 95% or so, so you either need to accept the need to build it in the last year or two, or pre-build it with foresight (and accept the resulting overcompliance in earlier years), or use DAC to offset it. I'm excited that our new approach seems to be working so well. I'm going to remove the old approach, clean up and organize the new code, and then get the demand shifting techs moved upstream of capacity construction.
jrissman
commented
1 year ago RPS foresight is now working (see issue #255 for some graphs). It helps the electricity system approach the RPS target more gradually, as intended. It doesn't help reduce the build-out much in the final year when the RPS value is going to 100%, though, as it complies with the values it sees by building whatever it would have built anyway to meet a given RPS percentage (for instance, it has to build certain sorts of things to go from 75% to 80%, which are different from the sorts of things it has to build to go from 95% to 100%). Foresight may mean it will do that sooner, but it won't tackle the difficult 95% to 100% segment sooner.
It should be thought of as climbing a staircase. Each stair is an RPS increment, say, 5%. Without foresight, we climb the first half of the staircase slowly, then climb the second half of the staircase quickly. With foresight, we climb the staircase at a more even pace throughout. As a result, we are taking some of the intermediate stairs earlier. But with or without foresight, we climb the last stair in the last year (since the goal of foresight is to smooth out the climb, not to reach the top earlier).
jrissman
commented
1 year ago After finishing the remaining steps for endogenous grid battery deployment and hooking that code into the model, I started looking into the power plant supply curves. I reviewed the tier-based implementation, and I must say that it is very clever! However, it is a lot of complicated code plus a new subscript, and it mandates a supply curve shape that looks like five stairsteps. I wanted to see if we could allow the user to define a cost curve using an unlimited number of points (to make an arbitrary shape), rather than mandating five stair steps, and to use simpler code that will be easier for people to understand and maintain.
The easiest place to try this was in the cost-effective capacity construction code. In figuring out how to do this, I noticed Vensim has a function called LOOKUP INVERT() that lets you enter a Y value for a lookup and Vensim returns the X value, the inverse of normal lookup behavior. This means we don't need to flip X and Y in our lookup variable relative to what feels most logical, which makes the code easier to understand. (If SDE doesn't already support LOOKUP INVERT, it should be very easy to add this function to SDE.)
For lookups, we want a supply curve that is monotonically increasing with no flat stairsteps, so I put in a supply curve that looks like this:
The X-axis is the share of last year (existing) capacity. The Y-axis is the capacity cost multiplier, i.e., the multiplier applied to the base cost of a plant type as a penalty for building more and more of that plant type in a given year. The model finds the capacity cost multiplier that makes profit = revenue (i.e., when marginal profit is zero), then does a LOOKUP INVERT() on this Y value to find the corresponding X value (% of exisitng capacity).
We multiply the X-value by last year capacity to convert it from a share of last year capacity into MW. This is the number of MW that is cost-effective to build. Finally, we multiply by a subscripted variable that specifies the share of cost-effective capacity that is actually built in a single year.
The two red-highlighted variables in the screenshot above will be based on input data files (.csv).
The new structure can produce results that are extremely similar to the results from the tier system. In the following graph, I'm comparing output (i.e., MW of cost-effective capacity to build) from the new and old methods. Note how close the lines of the same colors are to each other.
The new version can be tweaked either by changing the curve shape (which can alter what is deemed cost-effective and by how much) or by changing the share of cost-effective capacity that is built in a single year (which effectively scales the final results up or down but won't change what is deemed cost-effective or by how much).
I haven't yet looked at how tiers are used elsewhere (for instance, in the RPS/CES section), so I can't yet say how we might replace them with smooth curves there. One tricky thing might be ensuring we properly account for anything already built when determining the cost of additional capacity construction - we can't just start from the beginning of the curve again.
robbieorvis
commented
1 year ago Jeff, this is great! Glad you also think the prior approach was clever 😊. I am happy to have a different approach that utilizes a smoother curve and forgoes the additional subscript. That should definitely help with run time.
I’ll wait to fully review, but generally speaking it looks like a pretty good implementation. I would not be surprised if the results are somewhat different, since the curve does a better job varying prices. It is an improvement in accuracy.
The cost tiers are also used in the RPS/CES and reliability section and so the new curves will need to replace the tiers in those sections. I also had originally calculated the tier costs in the section calculating cost per unit new elec output, so you’ll have to see what I did there and how you want to modify it.
In terms of accounting for movement along the curve, I think you can just subtract the amount added in the prior section from the total that is being added.
For example, for the RPS/CES, you can subtract out the amount added from cost-effectiveness alone, which will require a higher CES to achieve the same amount of required deployment. Then, in the reliability based deployment, you subtract out everything built for cost-effectiveness and RPS/CES. There is a slight misalignment because the costs for the RPS/CES are calculated differently, but I think it’s fine and aligned with our staged capacity expansion approach.
There are a couple of other questions:
Finally, assuming we keep this structure, just flagging it is one more thing for Todd to add to SDE, though presumably it shouldn’t be very challenging and should just be very close to the LOOKUP function.
Looking forward to seeing it fully implemented!
From: Jeff Rissman @.> Sent: Friday, September 15, 2023 1:53 AM To: EnergyInnovation/eps-us @.> Cc: Robbie Orvis @.>; Mention @.> Subject: Re: [EnergyInnovation/eps-us] Move to hourly electricity dispatch (Issue #232)
After finishing the remaining steps for endogenous grid battery deployment and hooking that code into the model, I started looking into the power plant supply curves. I reviewed the tier-based implementation, and I must say that it is very clever! However, it is a lot of complicated code plus a new subscript, and it mandates a supply curve shape that looks like five stairsteps. I wanted to see if we could allow the user to define a cost curve using an unlimited number of points (to make an arbitrary shape), rather than mandating five stair steps, and to use simpler code that will be easier for people to understand and maintain.
The easiest place to try this was in the cost-effective capacity construction code. In figuring out how to do this, I noticed Vensim has a function called LOOKUP INVERT() that lets you enter a Y value for a lookup and Vensim returns the X value, the inverse of normal lookup behavior. This means we don't need to flip X and Y in our lookup variable relative to what feels most logical, which makes the code easier to understand. (If SDE doesn't already support LOOKUP INVERT, it should be very easy to add this function to SDE.)
For lookups, we want a supply curve that is monotonically increasing with no flat stairsteps, so I put in a supply curve that looks like this:
[Lookup_Var]https://user-images.githubusercontent.com/7120106/268181623-e0d9c954-4bc5-4330-b25d-dfb6c7aa12a8.png
The X-axis is the share of last year (existing) capacity. The Y-axis is the capacity cost multiplier, i.e., the multiplier applied to the base cost of a plant type as a penalty for building more and more of that plant type in a given year. The model finds the capacity cost multiplier that makes profit = revenue (i.e., when marginal profit is zero), then does a LOOKUP INVERT() on this Y value to find the corresponding X value (% of exisitng capacity).
We multiply the X-value by last year capacity to convert it from a share of last year capacity into MW. This is the number of MW that is cost-effective to build. Finally, we multiply by a subscripted variable that specifies the share of cost-effective capacity that is actually built in a single year.
[New_Structure_2]https://user-images.githubusercontent.com/7120106/268182991-7d4beff3-fc11-4d80-af47-cb985a239dab.png
The two red-highlighted variables in the screenshot above will be based on input data files (.csv).
The new structure can produce results that are extremely similar to the results from the tier system. In the following graph, I'm comparing output (i.e., MW of cost-effective capacity to build) from the new and old methods. Note how close the lines of the same colors are to each other.
[Output_Comparison]https://user-images.githubusercontent.com/7120106/268182266-c3803c1b-2ab3-43ba-88cb-47c6ced2e6f5.png
The new version can be tweaked either by changing the curve shape (which can alter what is deemed cost-effective and by how much) or by changing the share of cost-effective capacity that is built in a single year (which effectively scales the final results up or down but won't change what is deemed cost-effective or by how much).
I haven't yet looked at how tiers are used elsewhere (for instance, in the RPS/CES section), so I can't yet say how we might replace them with smooth curves there. One tricky thing might be ensuring we properly account for anything already built when determining the cost of additional capacity construction - we can't just start from the beginning of the curve again.
— Reply to this email directly, view it on GitHubhttps://github.com/EnergyInnovation/eps-us/issues/232#issuecomment-1720715217, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AK5N6SPKXNEZKASBPU6NJYLX2PUMXANCNFSM5W3M6AJQ. You are receiving this because you were mentioned.Message ID: @.**@.>>
ToddFincannonEI
commented
1 year ago @robbieorvis @jrissman SDEverywhere already has LOOKUP INVERT, so full steam ahead!
jrissman
commented
1 year ago The co-optimized RPS and cost-effective capacity construction Robbie made seems like a great solution. I've cleaned it up and integrated it into the calculation flow. It's working nicely.
But, in testing, I noticed a problem where the electricity sector as a whole is under- or over-dispatching slightly in years late in the 2040s with a 100% RPS. Look at the red line in the following graph, which is ideally supposed to stay centered in the thick orange line.
This doesn't appear to be related to the RPS, as this is over- or under-dispatching electricity as a whole, not an issue with hitting the RPS targets. (It's quite good at hitting the RPS target without going over or under, as you can see by comparing the blue and thick green lines above.) I traced the problem to the cost-effective dispatch section, which sometimes was doing a poor job of converging and therefore introducing visible error into the total amount to dispatch.
I thought I would try adding more optimization passes to see if this improved convergence. First, I updated the model to make it easy to adjust the number of optimization passes by editing the subscripts (so, all the variables' equations refer to a new subrange called last pass that can be updated in the subscripts pane to be whichever pass is last). This works.
Then I tried upping it from 20 to 30 passes. To my surprise, the convergence is far worse this way. I don't understand why increasing the number of optimization passes breaks it so severely.
I tried to debug it for a while, but it was getting late and I wasn't making headway, so I reverted to 20 optimization passes. I think we might want to examine the cost-effective dispatch step and convergence at some point prior to release to try to address issues like the one in the graph above. Ideally, the model should not break when you increase the number of optimization passes! But these are not the most critical things, and I need to move on, so I'm going to leave it be and work on replacing tiers with a smooth curve in the reliability code, then perhaps adding a supply curve to the grid battery code, and cleaning things up.
jrissman
commented
1 year ago Okay, I implemented smooth supply curves for grid battery deployment. The model now is able to deploy more batteries when there are a larger installed base of batteries, and in any given year, there are diminishing returns to profitability in terms of how many batteries can be built all in the same year. This reduces the spikiness of the old grid battery deployment method (that used a linear Capacity Response to Profits variable) and results in more realism. Here are graphs comparing the old and new data in the BAU scenario.
OLD version of grid batteries deployed this year
NEW version of grid batteries deployed this year
The bumps in the profitable years are still visible, but they no longer create immense spikes of deployment in those specific years.
Here is the same comparison for cumulative grid battery capacity deployed.
OLD version of cumulative grid batteries deployed
NEW version of cumulative grid batteries deployed
The old version is an unrealistic plateau, where it deploys most of the batteries in a single year, then deploys very few after that. The new version is smoother and shows faster growth near the end, once the battery-making industry is mature and scaled-up.
Note that the results might be even better if we didn't have one year that appears to be weirdly profitable for grid batteries. The new supply curve deals with that irregularity reasonably well, but we might ultimately not have such irregularity in grid battery profitability.
jrissman
commented
1 year ago Incidentally, seeing how badly linear such-and-such response to profits relationships tend to perform increases my skepticism toward elasticities in economics, since they are almost always reported as point elasticities (i.e., a linear slope, just like a such-and-such response to profits relationship). We might want to entertain the idea of using curves based on LOOKUPS wherever we currently use a linear elasticity someday. We'd probably not be able to find elasticity curves in the literature, but we could have a default curve shape we always use for elasticities, and we ensure the midpoint of any given curve (where percent change is zero) reflects the point elasticity we found in the literature for that specific elasticity.
robbieorvis
commented
1 year ago Thanks for all this Jeff! I will look at the dispatch issue today and tomorrow and see if I can pinpoint an issue/solution.
robbieorvis
commented
1 year ago Okay, the main source of the problem is the "initial guess at marginal dispatch cost" being too low. I tried various levels, but in short, this needs to be set to a sufficiently high level to allow the model to correctly solve the optimization loop. A value of 30 and 50 (I changed the max value from 1 to 30 and 50) eliminated unmet demand in almost every hour of every year, with the exception being a few hours in 2050.
For those few hours in 2050, I think there is something going on with the cost shifting from the CES. I'm looking into it. But the fix above solves 99.9% of the problem.
robbieorvis
commented
1 year ago Okay, what worked is keeping the existing structure, but subtracting the CES price (because it's a negative) from the initial guess price. See formula below. I'm not committing or pushing anything but suggest you try this and see if it fully solves the problem across different scenarios. It worked for me:
Initial Guess at Marginal Dispatch Cost= MAX(1-Adjustment in Case Minimum Dispatch Cost is Less than Zero,ZIDZ( SUM( Dispatch Cost per Unit Electricity Output[Electricity Source!]
Electricity Generation Capacity[Electricity Source!] ),
SUM(Electricity Generation Capacity[Electricity Source!]) ))
jrissman
commented
1 year ago Yes, this is a good fix. The initial guess is normally set as the average dispatch cost of the plant types, but in some years in the 100% RPS case, that was becoming negative, so it was hitting the MAX(1) term, but 1 is too small an initial guess. It takes forever for a guess staring at 1 to work its way up to a big enough step size to reach the price it's seeking.
I made the adjustment for Adjustment in Case Minimum Dispatch Cost is Less than Zero to both terms of the MAX function (the 1 and the weighted average), just so we're adjusting it consistently and don't have to rely on the 1 so much. I'm also thinking 1 might be too small as a minimum initial guess, so I will up it to 10, which is still much less than any marginal dispatch cost we see in any year, but will be able to reach larger positive values more quickly.
The resulting equation is:
MAX(
10 - Adjustment in Case Minimum Dispatch Cost is Less than Zero,
ZIDZ(
SUM(
Dispatch Cost per Unit Electricity Output[Electricity Source!]
* Electricity Generation Capacity[Electricity Source!]
),
SUM(Electricity Generation Capacity[Electricity Source!])
)
- Adjustment in Case Minimum Dispatch Cost is Less than Zero
)It converges beautifully now in every hour in both the BAU and 100% RPS cases (the only two cases I tested). It's actually remarkable how we've gradually improved this convergence algorithm's efficiency, so it is now able to achieve such good performance in just 20 passes across a wide range of input conditions.
jrissman
commented
1 year ago Based on Mike's feedback on our new hourly electricity dispatch graphs, I made the following small fixes and tweaks today:
These items were necessary to get proper-looking hourly dispatch graphs, which may be useful for @robbieorvis's EPS 4.0 presentation to staff next week. After Todd updates the web app, we should take new screenshots, unless you plan a live demo.
The one piece of feedback from Mike I could not implement today was separating battery charging from electricity exports, and separating battery discharging from electricity imports. This could be done but would require more Vensim work than I had time for.
I also began understanding the reliability section, added some output metrics so I can better see what it's doing, and made a few minor fixes but haven't put a smooth curve in yet. In the BAU case, it currently seems to think there is no unmet demand + reserve margin in years after 2021 but wants to build capacity anyway, which I will investigate further.
robbieorvis
commented
1 year ago Regarding reliability, I think the issue might be that you used last pass when in unmet reserve margin by hour, which is showing you the results after running through the optimization loop. So it shouldn't have any unmet demand. If you put pass1 in there instead, you'll see the years with a capacity price are (mostly) aligned with when there is a reliability need. There are a few years where the model is producing a capacity price when it's not needed, which is weird. I'm looking into that.
From: Jeff Rissman @.> Sent: Wednesday, September 27, 2023 9:28 PM To: EnergyInnovation/eps-us @.> Cc: Robbie Orvis @.>; Mention @.> Subject: Re: [EnergyInnovation/eps-us] Move to hourly electricity dispatch (Issue #232)
Based on Mike's feedback on our new hourly electricity dispatch graphs, I made the following small fixes and tweaks today:
These items were necessary to get proper-looking hourly dispatch graphs, which may be useful for @robbieorvishttps://github.com/robbieorvis's EPS 4.0 presentation to staff next week. After Todd updates the web app, we should take new screenshots, unless you plan a live demo.
The one piece of feedback from Mike I could not implement today was separating battery charging from electricity exports, and separating battery discharging from electricity imports. This could be done but would require more Vensim work than I had time for.
I also began understanding the reliability section, added some output metrics so I can better see what it's doing, and made a few minor fixes but haven't put a smooth curve in yet. In the BAU case, it currently seems to think there is no unmet demand + reserve margin in years after 2021 but wants to build capacity anyway, which I will investigate further.
- Reply to this email directly, view it on GitHubhttps://github.com/EnergyInnovation/eps-us/issues/232#issuecomment-1738313656, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AK5N6SJZUBMO3DH6UGLOI6DX4THDNANCNFSM5W3M6AJQ. You are receiving this because you were mentioned.Message ID: @.**@.>>
robbieorvis
commented
1 year ago I just pushed a few updates.
One of the issues with the evaluation previously is that you were looking at the reliability requirement after at least one round of optimization in the reliability pass. The starting initial guess is non-zero, meaning that you were seeing the demand for new resources for reliability after the model had already added a bunch on the first pass. I created a new reliability metric that shows the need for new reliability resources prior to the reliability optimization. This shows that the reliability mechanism is working extremely well, producing just the right amount of resources to meet reliability.
Relatedly, we had another issue similar to what we were seeing earlier with the CES, where any amount of reliability need would trigger significant builds, even if the CES was very close to zero. Implementing the cost curves and aligning the structure with the cost-effectiveness builds eliminated this problem. Now, if the capacity price is zero, the reliability calculation will calculate the same result as the cost-effectiveness calculations, and as part of the calculation we subtract out what is built in the cost-effectiveness loop, meaning that it nets to zero.
The reliability feature seems to be working extremely well for the moment, and I think it may be complete, but Jeff should review before we close that piece.
From: Jeff Rissman @.> Sent: Wednesday, September 27, 2023 9:28 PM To: EnergyInnovation/eps-us @.> Cc: Robbie Orvis @.>; Mention @.> Subject: Re: [EnergyInnovation/eps-us] Move to hourly electricity dispatch (Issue #232)
Based on Mike's feedback on our new hourly electricity dispatch graphs, I made the following small fixes and tweaks today:
These items were necessary to get proper-looking hourly dispatch graphs, which may be useful for @robbieorvishttps://github.com/robbieorvis's EPS 4.0 presentation to staff next week. After Todd updates the web app, we should take new screenshots, unless you plan a live demo.
The one piece of feedback from Mike I could not implement today was separating battery charging from electricity exports, and separating battery discharging from electricity imports. This could be done but would require more Vensim work than I had time for.
I also began understanding the reliability section, added some output metrics so I can better see what it's doing, and made a few minor fixes but haven't put a smooth curve in yet. In the BAU case, it currently seems to think there is no unmet demand + reserve margin in years after 2021 but wants to build capacity anyway, which I will investigate further.
- Reply to this email directly, view it on GitHubhttps://github.com/EnergyInnovation/eps-us/issues/232#issuecomment-1738313656, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AK5N6SJZUBMO3DH6UGLOI6DX4THDNANCNFSM5W3M6AJQ. You are receiving this because you were mentioned.Message ID: @.**@.>>
robbieorvis
commented
1 year ago One final note: I needed to disable the economic feedback loops for now. I think we should keep these off until we build out the BAU side of everything, then check to make sure it's working correctly. There was feedback issue with cargo distance transported when I turned back on the IRA incentives.
From: Jeff Rissman @.> Sent: Wednesday, September 27, 2023 9:28 PM To: EnergyInnovation/eps-us @.> Cc: Robbie Orvis @.>; Mention @.> Subject: Re: [EnergyInnovation/eps-us] Move to hourly electricity dispatch (Issue #232)
Based on Mike's feedback on our new hourly electricity dispatch graphs, I made the following small fixes and tweaks today:
These items were necessary to get proper-looking hourly dispatch graphs, which may be useful for @robbieorvishttps://github.com/robbieorvis's EPS 4.0 presentation to staff next week. After Todd updates the web app, we should take new screenshots, unless you plan a live demo.
The one piece of feedback from Mike I could not implement today was separating battery charging from electricity exports, and separating battery discharging from electricity imports. This could be done but would require more Vensim work than I had time for.
I also began understanding the reliability section, added some output metrics so I can better see what it's doing, and made a few minor fixes but haven't put a smooth curve in yet. In the BAU case, it currently seems to think there is no unmet demand + reserve margin in years after 2021 but wants to build capacity anyway, which I will investigate further.
- Reply to this email directly, view it on GitHubhttps://github.com/EnergyInnovation/eps-us/issues/232#issuecomment-1738313656, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AK5N6SJZUBMO3DH6UGLOI6DX4THDNANCNFSM5W3M6AJQ. You are receiving this because you were mentioned.Message ID: @.**@.>>
robbieorvis
commented
1 year ago Sorry ignore that last comment - it was related to a data export error I made. We can keep the feedback loops on.
From: Jeff Rissman @.> Sent: Wednesday, September 27, 2023 9:28 PM To: EnergyInnovation/eps-us @.> Cc: Robbie Orvis @.>; Mention @.> Subject: Re: [EnergyInnovation/eps-us] Move to hourly electricity dispatch (Issue #232)
Based on Mike's feedback on our new hourly electricity dispatch graphs, I made the following small fixes and tweaks today:
These items were necessary to get proper-looking hourly dispatch graphs, which may be useful for @robbieorvishttps://github.com/robbieorvis's EPS 4.0 presentation to staff next week. After Todd updates the web app, we should take new screenshots, unless you plan a live demo.
The one piece of feedback from Mike I could not implement today was separating battery charging from electricity exports, and separating battery discharging from electricity imports. This could be done but would require more Vensim work than I had time for.
I also began understanding the reliability section, added some output metrics so I can better see what it's doing, and made a few minor fixes but haven't put a smooth curve in yet. In the BAU case, it currently seems to think there is no unmet demand + reserve margin in years after 2021 but wants to build capacity anyway, which I will investigate further.
- Reply to this email directly, view it on GitHubhttps://github.com/EnergyInnovation/eps-us/issues/232#issuecomment-1738313656, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AK5N6SJZUBMO3DH6UGLOI6DX4THDNANCNFSM5W3M6AJQ. You are receiving this because you were mentioned.Message ID: @.**@.>>
robbieorvis
commented
1 year ago Adding some notes from slack here identifying a few issues (I have been testing the models with state data which has helped to flag a few things:
From: Robbie Orvis Sent: Thursday, September 28, 2023 11:02 AM To: 'EnergyInnovation/eps-us' @.>; EnergyInnovation/eps-us @.> Cc: Mention @.***> Subject: RE: [EnergyInnovation/eps-us] Move to hourly electricity dispatch (Issue #232)
Sorry ignore that last comment - it was related to a data export error I made. We can keep the feedback loops on.
From: Jeff Rissman @.**@.>> Sent: Wednesday, September 27, 2023 9:28 PM To: EnergyInnovation/eps-us @.**@.>> Cc: Robbie Orvis @.**@.>>; Mention @.**@.>> Subject: Re: [EnergyInnovation/eps-us] Move to hourly electricity dispatch (Issue #232)
Based on Mike's feedback on our new hourly electricity dispatch graphs, I made the following small fixes and tweaks today:
These items were necessary to get proper-looking hourly dispatch graphs, which may be useful for @robbieorvishttps://github.com/robbieorvis's EPS 4.0 presentation to staff next week. After Todd updates the web app, we should take new screenshots, unless you plan a live demo.
The one piece of feedback from Mike I could not implement today was separating battery charging from electricity exports, and separating battery discharging from electricity imports. This could be done but would require more Vensim work than I had time for.
I also began understanding the reliability section, added some output metrics so I can better see what it's doing, and made a few minor fixes but haven't put a smooth curve in yet. In the BAU case, it currently seems to think there is no unmet demand + reserve margin in years after 2021 but wants to build capacity anyway, which I will investigate further.
- Reply to this email directly, view it on GitHubhttps://github.com/EnergyInnovation/eps-us/issues/232#issuecomment-1738313656, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AK5N6SJZUBMO3DH6UGLOI6DX4THDNANCNFSM5W3M6AJQ. You are receiving this because you were mentioned.Message ID: @.**@.>>
jrissman
commented
1 year ago Hi Team,
Today I completed a few things:
Two things to note:
First, I think this will solve the issue you were seeing with electricity imports and exports not being handled well in the state-level models. It also breaks out electricity imports and exports as separate data series on hourly dispatch graphs, no longer grouped with storage, which addresses Mike's last piece of feedback on those graphs.
Note that there is a new input variable, SoEIaEbH Share of Electricity Imports and Exports by Hour. It is formatted like SHELF, so it should be familiar. Right now, it contains stand-in data that says electricity imports and electricity exports are evenly spread across every hour of the year. This is very unrealistic stand-in data, as it says the region is simultaneously importing and exporting electricity in every hour. I strongly suggest we replace the stand-in input data in SoEIaEbH with something more realistic.
--
Second, I noticed that the model now sometimes exceeds the RPS target, as in the following graph.
I kept looking at the RPS construction code and the dispatch code and couldn't find anything wrong with either. Finally, I realized that they are both working right. It's just that we have the reliability-based construction enabled now. It operates after RPS-based construction and sometimes chooses to build RPS-qualifying resources like wind and solar for reliability, after the RPS system built as many renewables as we needed to meet the RPS. This can lead to exceeding the RPS target in some years.
I think it may be fine to leave it like this. It may be that a utility would build enough renewables to meet an RPS, then realize that renewables can be the cheapest way to meet some yet-unmet reliability needs, and build even more. Why not?
But if you are strongly concerned this over-compliance may be incorrect, we might need to integrate reliability-based construction into the same optimization loop as cost-effective and RPS-based construction, so it can co-optimize everything all together. But I'm nervous about that because I'm not sure how well it would handle the dual targets (the RPS target and the reliability target), with each of them increasing or decreasing its own credit value (CES credit and capacity credit). Within each optimization pass, there would still need to be an ordering, presumably RPS before reliability. I don't know if it would readily settle down to a stable solution that meets reliability while narrowly hitting the RPS, or if we might get oscillation where the CES and capacity credit prices keep influencing one another and prevent convergence, or if it settles down to a stable solution and we discover that it still over-complies with the RPS because it's legitimately the optimum solution (!). Hard to say without building it. I don't think we should do this for EPS 4.0.0 unless you start to see RPS over-compliance due to reliability construction that is large and clearly unrealistic.
--
I didn't have time to get to CCS retrofitting today. I also didn't address the new issue you found about the "fallback dispatch cost when using only guaranteed dispatch"
Unfortunately, I've already spent two more days on EPS model dev this week than I had planned for, and I simply must switch back over to Industry work, as there is a large paper I need to edit by COB Tuesday. I've been working late every night and am getting burned out. Maybe you can try integrating RPS retrofitting into the RPS optimization loop and get it working, and I'll be happy to review it later as soon as I have the time.
jrissman
commented
1 year ago Also note that right now, CCS retrofits (which are disabled) are being added to newly built capacity, but that's going to act just like building a new CCS-equipped plant type. That is okay for finding the right amount to hit an RPS target, but it won't handle the vintage tracking correctly. We could potentially continue to insert retrofits into new-built capacity if, downstream, we (a) implement a discount on their construction cost because it was a retrofit rather than a new build, and (b) we add a retirement of the equivalent number of MW of capacity of the non-CCS version of that plant type. The simplest thing to do is to let the existing system for identifying retirements by vintage decide which old vintages to remove (it will take the oldest), even though that might not correspond to which plants are actually the ones most likely to be retrofit. But the model has no way to know which plant might get retrofit, and it doesn't track plants individually anyway.
Finally, the model works fine without CCS retrofits, so we can still consider them an optional stretch goal for EPS 4.0.0.
This builds on the discussion of other electricity sector improvements in issues #219 and #106 and logs as an issue conversations we've been having already about electricity dispatch.
We should consider moving to an hourly dispatch model to improve dispatch and reliability modeling. I completed a proof of concept of this approach in #106 that can be easily implemented and does not appear to result in a significant increase in runtime.
In evaluating the California EPS it has become very obvious that this improvement is extremely important to evaluate scenarios of deep electricity sector decarbonization with any level of confidence. This is because solar is the dominant low carbon electricity source dispatched, but when evaluating the hourly load requirements and solar availability, it is clear that the current approach of looking at peak annual seasonal demand and resource availability does not capture the reliability limitations of a power system. For example, in the CA model, solar has a pretty high capacity factor during peak hours, but it a zero capacity factor in later hours of the day when demand is high but not at peak levels. Using an annual approach, the model assumes that all demand can be met and even that peak is satisfied. But the real issue is the requirement to meet net demand after removing variable generation. To get that right, we need some representation of hourly load and resource availability. Because the implementation of an hourly modeling approach in #106 proved viable and fast, I just suggest we use this approach.
In estimating reliability, the model could look at load in a year and existing resources and do a modeling pass to see if there is a shortfall in any given hour, after applying the reserve margin. This is likely to result in need for additional resources to cover hours where there is a large net load requirement (net load = demand - variable renewable generation). In California, this is often in the evening hours, which is what I anticipate the modeling will show.
Moving to hourly dispatch will do a much better job representing system constraints and resource availability, improving our ability to model reliability capacity requirements while also allowing us to do economic capacity expansion as outlined in #106. It does likely require us to move back to ALLOCATE AVAILABLE, but as discussed elsewhere, this is required anyway given the limitation of the logit functions to model physical constraints.