EnergyInnovation / eps-us

Energy Policy Simulator - United States
GNU General Public License v3.0
22 stars 7 forks source link

Use subregions for ZEV standards #256

Closed jrissman closed 2 years ago

jrissman commented 2 years ago

From Robbie:

In light of the work on the IRA and in reviewing the great news out of California today, I’ve been thinking about the way we are modeling the ZEV sales standard in the EPS and I think it might be too conservative. Moving it more towards how we model the RPS policy is potentially an improvement.

Right now, we model the ZEV standard by first letting the model just do a normal new vehicle allocation, then checking to see if the new sales shares comply, and if not, forcefully reallocating some of the non-ZEV vehicle to ZEVs, such that we meet the minimum standard.

The issue with this approach is that it doesn’t account for sub-regional detail and therefore likely understates ZEV deployment, because it offsets the compliance for ZEV regions with lower EV sales than the modeling projects based on cost in other regions.

The attached has an example of this. Policy Allocation Test.xlsx

The first set of calculations mimics what we do now. Allocating just based on costs and using the logit, the model allocates 33% to ZEV. Then, because this is below the weighted average value of 35% (100% ZEV requirement for states that make up 35% of new vehicle sales), we manually adjust to get to 35% and 65% for each of EVs and ICEs. But another way of interpreting this is that the ZEV mandate regions are in compliance, and the non-ZEV mandate regions sell 0% ZEVs. This is clearly wrong. What we should actually be doing is applying the ZEV mandate to the share of new vehicle sales covered, then letting the model do its normal cost based allocation on the remaining sales. When we take this new approach, instead of getting 35% ZEVs, we get 56.67% ZEVs, because the model is first making sure we comply with the regions with the ZEV mandate, and then running the cost based allocation on remaining sales. This does assume ZEV mandate and non ZEV mandate states are the same otherwise, but that is an assumption that is true for other places in the model as well.

This approach is a much better way to capture sub-regional policy in the BAU case in the EPS, which will become increasingly important in the US and perhaps elsewhere (China provinces, EU countries, etc…). It is also much closer to the approach we use for the RPS policy in the electricity sector, where we have one pass that makes sure the RPS is met (and because the RPS covers a share of electricity sales, it is very similar to the concept of covering a share of new vehicle sales), then we let the model do a normal allocation pass on the residual electricity sales.

For the proposed method for ZEVs, we would need a new variable with the BAU percentage of new vehicle sales covered by ZEV standard. This can be approximated by looking at vehicle registrations in states with ZEV standards and dividing by total vehicle registrations (I looked earlier and new vehicle sales percentages hew pretty closely to a state’s share of total vehicle registrations). When using the policy levers, the user setting would need to apply to 100% of the market, representing the region of the EPS.

jrissman commented 2 years ago

From me:

This is a great observation, and what you say below is compelling. But the logit function’s first pass doesn’t represent a share of regions that are in 100% compliance. It represents consumer preferences/choices across the entire modeled region. So if the first pass says that without a ZEV standard, 35% of new vehicle sales would be ZEVs, this says that 35% of buyers of new vehicles voluntarily choose ZEVs across the whole United States (or whatever region the EPS is modeling, but let’s use the U.S. as our example in this email).

It would be very unlikely if all of those buyers were concentrated in particular U.S. states. Most likely, the 35% of total buyers who want ZEVs are spread out across the country, with at least a few in each of the 50 states. Intuitively we know there may be greater concentrations in some areas than others (such as cities vs. rural areas), but we don’t model that sub-regional detail. In essence, the model assumes there is one region, so it doesn’t matter how people are spread out throughout that region, and if the natural 35% sales is lower than the ZEV requirement, it reallocates to meet the national ZEV requirement. There would be no justification for exceeding it because the natural tendency of the buyers in this example was to not meet the requirement.

If we assume the 35% entirely comes from sub-regions meeting sub-regional targets, and we allocate all the remainder, then we would overstate ZEV sales, because some of the 35% we initially allocated actually consisted of buyers in regions that are not meeting their targets, like some buyers who choose EVs even though their sub-region has no EV requirement.

So our current method works well for single, homogenous regions without sub-regional policies. That might be good when modeling a single U.S. state. But when modeling the entire U.S., where there can be policies at sub-regional scale, you are correctly pointing out that we might be under-estimating ZEV deployment. However, I think your proposed fix goes too far in the opposite direction and would overstate ZEV deployment. We should try to find something in the middle.

If we aren’t doing too many math operations at sub-regional scale, we could introduce a subscript for sub-region, each of which can have its own ZEV percentage requirement. Then we keep the current approach (which first allocates based on pure consumer preferences, then shifts sales if necessary to meet the target), but do it separately in each sub-region, so that a sub-region with a 100% ZEV mandate like California would be properly represented, without affecting the number of EV buyers in, say, South Carolina. I think that’s feasible here, since the only thing varying at sub-regional scale is the ZEV mandate, so we just do the shift 50 times instead of 1 time, which should be quick enough not to have any meaningful runtime impact.

jrissman commented 2 years ago

From Robbie:

If I understand correctly, I think we are essentially converging on the same thing, but (a thought) rather than have multiple regions, we might be able to do it with just two regions: states with and without ZEV rules. If we can do it this way, we don’t need to add a subscript. We would do two passes, first one for states with ZEV rules and then one without. For the states with ZEV rules, we keep the current approach where we allow a cost-based pass, then adjust as needed to comply with the target. For the second pass for remaining states, we just let the model do a normal cost pass.

The example I gave in the spreadsheet is functionally equivalent, I think, because it looked at case where all the states with ZEV standards had a 100% target. But as soon as that’s less than 100%, my approach would overcompensate, to your point.

To sum, I think your approach is right, but I wonder if we can just use percentages of coverage of sales rather than trying to add a new subscript for multiple regions?

jrissman commented 2 years ago

From me:

If every state has its own unique percentage, in theory, we’d want one bucket for every unique percentage (though each percentage could be shared by multiple states). But if it’s a bunch of states with 100% requirements and a bunch of other states with 0% requirements, then two buckets is enough.

Rather than conceive of buckets as sub-regions, we could conceive of them as ZEV stringency tiers. The simplest approach has just a 0% and a 100% tier, but if you want more detail, we could do something like eleven tiers (0%, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, 100%) and put each sub-region (U.S. state) into the closest tier. The nice thing about that is that it scales up to any number of sub-regions without impacting model run speed, because you’re summing them into eleven tiers in Excel before you run calculations in Vensim. And with the tiers only separated by 10% gaps, any loss in accuracy would be imperceptible.

That might be helpful in modeling a transition toward 100% EV mandates, where there will be intermediate percentages in various years, as well as the possibility of some purple states enacting some ZEV policies but stalling out before they reach 100%.

jrissman commented 2 years ago

From Megan:

Thanks for sharing this, Robbie. I like the idea of this kind of update, and Jeff’s approach makes sense to me as a way of capturing variation without having to add an unmanageable number of subscripts.

One other point to consider is that our approach currently sets a standard for battery electric vehicles only, but CA’s standard also applies to plug-in hybrids (and I believe hydrogen vehicles, which will be a much smaller share). If we’re implementing this change, I think we’d likely want to add the ability to specify which vehicle technologies contribute to meeting the standard when calculating the first pass.

jrissman commented 2 years ago

Hey, @robbieorvis, quick question on the implementation of a policy lever for ZEV standards after dividing vehicle sales into stringency tiers.

Suppose for the U.S. national model we have 50 states with 50 different ZEV percentage requirements in a given year, and we group them into tiers like 0%, 10%, 20%... 90%, 100%. Then the user specifies a ZEV standard in that year of 40%.

Do you want me to pull up all the states that are below 40% so that their standards are 40% without changing the standards of the states that are already exceeding 40% in that year? This is the most straightforward way. But it would over-comply with the standard on a nationwide average basis.

Or do you want me to take a weighted average of the car sales in the eleven stringency buckets (suppose that average comes out to 33%), then decide how to allocate increases so that a nationally weighted average of 40% is achieved? It's not obvious how to allocate these increases. One option would be to allocate the required increase to all states in proportion to how far their current requirement is from 100%, but this might be seen as unfair because states that have a goal of 80% in that year might argue that they shouldn't need to further increase their target. Perhaps a better option is to allocate the increase to all states that are below 40% in proportion to how far their targets are from 40% in that year. This is similar to the first option but much more complex to calculate because you're only increasing their targets enough to reach a nationally weighted average of 40% rather than increasing all the laggard states' targets to 40%.

Note that we'd actually be working with the eleven stringency buckets and the amount of cars sold that are in each bucket, not actually 50 states, but I used "states" in the description above to make it more readable. It should be understood to actually refer to buckets of states.

Also note that the latter approach would require complicated new input data that says how many vehicles are sold in each stringency bucket in each year (which can change as states move between buckets) in order to perform the weighting, and BAU data for that variable would not necessarily remain valid in policy scenarios (because states would move between buckets in different years than in the BAU dataset), so we probably would need to actually track 50 states rather than 11 stringency buckets if we wanted to perform the weighting accurately. Which, I suppose, raises the question of why we'd use stringency buckets at all. Maybe we just introduce a sub-region subscript with 50 elements and work with that.

Given the greater complexity and input data requirements of handling the nationally weighted average, it would be a whole lot easier if we could define the ZEV standard policy lever as the minimum required ZEV sales percentage in each state, rather than the minimum required nationally weighted average ZEV sales percentage. Is that okay?

robbieorvis commented 2 years ago

Hi Jeff, I’m not sure I fully follow, but here’s what I’m thinking. I just had to do this in Excel, replicating the logit functions, for some other modeling with some external partners. I will upload a copy of that file for you to see.

In the spreadsheet modeling I did, my approach was:

  1. Calculate the ZEV sales shares in each year of each state with a ZEV program. For example, if California has a 65% percent ZEV requirement in 2035 and it’s 10% of US sales, we would calculate a value of 6.5%. We do this for each state and sum the values (I have this data already).
  2. Calculate the non-ZEV sales share in each year of each state with a ZEV program. For example, in the same California example above, this would be (1-65%)*10%, or 3.5%.
  3. Calculate the share of states without ZEV program. Following the above example. If only California had a ZEV program, this would be (1-10%), or 90%.
  4. Then I did three allocations, because I actually changed this to a ZEV standard instead of a split EV, PHEV, and hydrogen standard (we might consider that with this update, including a boolean input file that flags which vehicles qualify… this would make the policy a lot like the RPS):
    • First pass: estimate the breakdown of vehicle sales shares for ZEV qualifying vehicles for ZEV states (i.e. set the share-weights to zero for non-qualifying technologies)
    • Second pass: estimate the breakdown of vehicle sales share for non-ZEV qualifying vehicles for ZEV states (i.e. set the share-weights to zero for the qualifying technologies)
    • Third pass: estimate the breakdown for vehicle sales shares for non-ZEV states
  5. Then, the total sales by vehicle type takes the percentages calculated in a 1-3 and multiplies them by each pass. The assumption is that states do not overcomply with the ZEV standard. Total market share by vehicle type =
    • First pass values * the percentage of sales nationally that are ZEV required, i.e. calculation in (1) PLUS
    • Second pass values * (1-the non ZEV percentage in states with ZEV standards), i.e. calculation in (2) PLUS
    • Third pass * share of sales from non-ZEV standard states, i.e. calculation in (3)

That is all just to get BAU values. Now, I think your question also pertains to how to handle situations where a policy comes into place on top of this. My feeling is that if the ZEV standard set by the user is less than the values calculated above, it is non-binding. Because the model is single region and we are looking at national policy, I think it should drive additional adoption on top of the sum of the state policies. I believe that is the easier approach, but not entirely sure based on your prior comment. It’s also basically the same approach to the RPS, where a setting that is too low fails to drive incremental adoption.

I hope that answers your question, but let me know if not. I will upload the file I am talking about for you to review as well.

robbieorvis commented 2 years ago

Here is the file I mentioned 20221116_ICCT IRA Modeling v4.xlsx

Note also: this approach moves the complexity IMO into the input data and out of Vensim. All we need are data on which vehicles would qualify (if you want to use a ZEV instead of just the EV standard approach; if not then you don't even need this because you don't need the first allocation pass... but in the long run we need to move to this type of approach because of how vehicle standards are designed), and the percentages for the scope of sales covered by ZEV requirements and the share of sales in states with ZEV standards that are not ZEV. subtracting these two from 1 give you the remaining value for states not covered by ZEV programs.

jrissman commented 2 years ago

@robbieorvis - thanks for going over the BAU approach and uploading the Excel file.

The idea of doing two passes, a ZEV-required pass and an unrestricted pass (for the remaining newly sold vehicles), similar to the RPS-qualifying and unrestricted passes in the electricity sector, makes sense to me. I can implement it that way. I don't think we need three passes because any states with no legislative ZEV requirement can have 0% of their demand in the ZEV pass and 100% of their demand in the unrestricted pass.

There are really only two wrinkles to iron out:

The Excel sheet you uploaded is more complex (and more specific to the idiosyncrasies of the Inflation Reduction Act) than we would want for core structure in the EPS. We can't move stuff into Excel unless the results are not affected by the policy levers, or we're making a parameter like deaths-per-unit-emissions that we use in the EPS. Fortunately, 50 allocations shouldn't be bad. Logit functions are fast because they are single-pass (unlike ALLOCATE AVAILABLE), and also, 50 allocations is a lot less than we get in the electricity sector if we try to do one for every hour of the year (8760). So I'm not worried about runtime impacts in this case.

I think I know what I need to do to move ahead with this using a new "subregion" subscript with 50 state elements (or maybe up to 56 elements if we have vehicle sales and ZEV standard data for Washington D.C. and for some or all of the five inhabited U.S. territories).

robbieorvis commented 2 years ago

Hi Jeff,

Thanks for looking at this. Sorry that Excel doc is confusing. It is indeed trying to do more than we want (and to cover the IRA). The main area to look is the ACCII tab and the first two passes for the passenger vehicle allocations.

A few responses/comments:

robbieorvis commented 2 years ago

One potential thought: we could change the input data to be: share of national sales covered by ZEV and weighted average ZEV target. I think that would give us everything needed to calculate the percentages for all three passes (sum of state sales ZEV amount, sum of state sales (1-ZEV amount), and share of total national sales not covered by ZEV.

jrissman commented 2 years ago

Okay, I think I understand your thinking about the three passes now. Your vision for the second pass is to not allow any ZEVs to be sold in that pass, to prevent over-compliance at the state level. That is different from the electricity sector's RPS, which has only two passes and allows more wind or solar to be built in the unrestricted (second) pass, if renewables are the most economical option.

I agree with you that two passes like the RPS at national scale doesn't work well here because, as you say, it would lead to systematic over-compliance nationally. Got it. 😊

However, I think it is problematic to not allow any individual state to exceed its ZEV mandate in ZEV sales percentage. There could be a state, province, country, or region that aims to use other policies, like subsidies, to boost EV sales with only a weak mandate as a floor, so allowing over-compliance would be critical. Even in the U.S., there can be over-compliance in states when standards are still weak, such as in early years while a state standard is just beginning to phase in. For example, if a state right now has no ZEV mandate but institutes one in the future, that state will be in over-compliance in the early years of their mandate because the state will already have some (perhaps many) EVs on the roads.

Suppose a state implements a ZEV mandate of 0.01% that stays constant through 2050. With the three-pass approach above, that state would have near-zero EV sales through 2050 because all of the sales would be funneled into the second pass, which disallows EVs.

So, I think we shouldn't have a "special case" for states with no standards that is handled differently from states with weak standards. To avoid bugs and problems with edge cases and early years of policy phase-in, we should have a methodology that operates in a smooth, continuous way for states that range from no EV standard all the way up to a 100% EV standard. I think we should avoid over-compliance only in cases where it is not economical to over-comply.

The current approach allocates everything freely, then increases the share of ZEVs if the free allocation didn't meet the standard. I think this is still broadly correct and does not overcomply. As I understood it, the issue with the current approach is whether, when done nationally, it might understate ZEV deployment because it allows states that are over-complying with their ZEV standards (or that have ZEVs but have no ZEV standard) to "make up for" some states that are actually missing their ZEV standards. If we model every state separately, we can ensure no state is missing its ZEV standard. Is that still the core issue?

Then, as a separate matter, we could allow multiple vehicle technologies to qualify for a ZEV standard, set by a flag, similar to RPS-qualifying technologies. We allocate amongst qualifying vehicle types when determining which type of ZEVs to add when we need to increase ZEVs in a state to meet that state's target.

So in summary:

I see no way around doing this with a "subregion" subscript at this point because, as I understand it, the core inaccuracy we want to address is the way that states that are over-complying with their ZEV standards (or that have ZEVs but have no ZEV standard) can "make up for" some states that are actually missing their ZEV standards. I don't see how to get that state-level resolution without a sub-region subscript.

If you think this situation is so rare that it never happens or doesn't matter, then I think the existing structure is already accurate, except for not yet including the customizable definition of which vehicle technologies qualify as ZEVs.

robbieorvis commented 2 years ago

Had one more thought: we might be able to solve overcompliance by just using a max function of the logit results. If the third allocation produces shares greater than the sum of the first two, then we just use the third pass for the states with ZEV.

Sent from my iPhone

On Nov 17, 2022, at 9:11 PM, Jeff Rissman @.***> wrote:



Okay, I think I understand your thinking about the three passes now. Your vision for the second pass is to not allow any ZEVs to be sold in that pass, to prevent over-compliance at the state level. That is different from the electricity sector's RPS, which has only two passes and allows more wind or solar to be built in the unrestricted (second) pass, if renewables are the most economical option.

I think it is problematic to not allow any individual state to exceed its ZEV mandate in ZEV sales percentage. There could be a state, province, country, or region that aims to use other policies, like subsidies, to boost EV sales with only a weak mandate as a floor, so allowing over-compliance would be critical. Even in the U.S., there can be over-compliance in states when standards are still weak, such as in early years while a state standard is just beginning to phase in. For example, if a state right now has no ZEV mandate but institutes one in the future, that state will be in over-compliance in the early years of their mandate because the state will already have some (perhaps many) EVs on the roads.

Suppose a state implements a ZEV mandate of 0.01% that stays constant through 2050. With the three-pass approach above, that state would have near-zero EV sales through 2050 because all of the sales would be funneled into the second pass, which disallows EVs.

So, I think we need to allow over-compliance even in U.S. states with standards. We shouldn't have a "special case" for states with no standards that is handled differently from states with weak standards. To avoid bugs and problems with edge cases and early years of policy phase-in, we should have a methodology that operates in a smooth, continuous way for states that range from no EV standard all the way up to a 100% EV standard. I think we should avoid over-compliance only in cases where it is not economical to over-comply, on the basis of the underlying economics and other policies like an EV subsidy, then over-compliance is the correct result.

The current approach allocates everything freely, then increases the share of ZEVs if it didn't meet the standard. I think this is still broadly correct and does not overcomply. As I understood it, the issue with the current approach is whether, when done nationally, it might understate ZEV deployment because it allows states that are over-complying with their ZEV standards (or that have ZEVs but have no ZEV standard) to "make up for" some states that are actually missing their ZEV standards. If we model every state separately, we can ensure no state is missing its ZEV standard. Is that still the core issue?

Then, as a separate matter, we could allow multiple vehicle technologies to qualify for a ZEV standard, set by a flag, similar to RPS-qualifying technologies. We allocate amongst qualifying vehicle types when determining which type of ZEVs to add when we need to increase ZEVs in a state to meet that state's target.

So in summary:

I see no way around doing this with a "subregion" subscript at this point because, as I understand it, the core inaccuracy we want to address is the way that states that are over-complying with their ZEV standards (or that have ZEVs but have no ZEV standard) can "make up for" some states that are actually missing their ZEV standards. I don't see how to get that state-level resolution without a sub-region subscript.

If you think this situation is so rare that it never happens or doesn't matter, then the existing structure is already accurate, except for not yet including the customizable definition of which vehicle technologies qualify as ZEVs.

— Reply to this email directly, view it on GitHubhttps://github.com/EnergyInnovation/eps-us/issues/256#issuecomment-1319456924, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AK5N6SMHB6ZPWZZWTTYBT4LWI3QVXANCNFSM6AAAAAARRO3SO4. You are receiving this because you were mentioned.Message ID: @.***>

jrissman commented 2 years ago

Yes, that seems like a reasonably clean way to structure the code flow. It's probably a bit cleaner than the approach outlined in my bulleted summary above! 😊

But I think it still has to be done at the subregion (state) level if we want to prevent the issue where some states can miss their targets if other states over-comply.

robbieorvis commented 2 years ago

Check out this example I put together, attached. I think the situation you are describe is avoided by using the MAX approach:

1) If we use the MAX system I described, then ZEV-standard states will always have at least as high of a sales share as the non-ZEV states, because whenever a national ZEV standard or incentives cause greater shares in the allocation than the the ZEV standard, the allocation will use the greater of the two.

2) There cannot be a case where, due to other states over-complying, ZEV states can undercomply, because of the dynamic above, unless I am misinterpreting your point or failed to calculate something.

Maybe I'm just not seeing it, but based on the three region example I just put together, it looks like we won't need a region subscript the single- and multi-region values are identical. ZEV Example.xlsx

robbieorvis commented 2 years ago

One other thing I learned this morning. We may indeed only need 1 allocation, because the ratios are the same between technologies across the different allocations I mentioned. For example, even if you have an allocation across all technologies that gets you 20% BEV, 5% PHEV, and 75% ICE, if you just did an allocation with the ZEV qualifying BEVs and PHEVs you would get 80% BEV and 20% PHEV (a ratio of 4:1). So we can just do a single allocation and scale the results to the qualifying technologies, which makes things simpler than have to program multiple allocations.

robbieorvis commented 2 years ago

Updated file for your review ZEV Example.xlsx

jrissman commented 2 years ago

Completed in 17ea4c6. As we discussed today, we're using sub-regions now, not stringency buckets nor a nationwide weighted average. The code is very clean and should be largely self-explanatory when examined. The EV minimum and hydrogen vehicle minimum policies were replaced with the new ZEV standard policy. I am using real data I found, not stand-in data, but nonetheless someone should look over trans/BRZSPbS just to ensure accuracy before releasing.

This entailed a lot of changes to a lot of different files. It needs its own release and must not be included in 3.4.3. This new sub-region feature is irrelevant to the U.S. state models because U.S. states don't have sub-regions with their own, varying ZEV standards. Trying to release them on this platform would be a huge amount of work to update 40+ files in each U.S. state repo for zero benefit in terms of model capabilities. Therefore, I've tagged the release prior to commit 17ea4c6 as 3.4.3 and would like you to release that tagged commit as 3.4.3 (updating the Version History and Download links in the docs accordingly). This EVs issue only pertains to the U.S. national model and can be released as a 3.4.4 release, which will not be the platform used by the U.S. state models.

Note that because the new ZEV standard only applies to road vehicles, when we removed the EV and H2 vehicle minimum policies, we lost the ability to require new non-road vehicles to be electric or to use hydrogen, such as rail electrification or hydrogen ships. We might consider putting back some policies mandating fuel shifting for non-road vehicles when we do a larger transportation overhaul. But that should not hold up 3.4.4, and should not come before we finish the electricity sector updates in 3.5.

In fact, this really must be the last feature we put in the 3.4 series because it is getting to be far too burdensome to port features into 3.5.