Closed jrissman closed 2 years ago
From me:
This is a great observation, and what you say below is compelling. But the logit function’s first pass doesn’t represent a share of regions that are in 100% compliance. It represents consumer preferences/choices across the entire modeled region. So if the first pass says that without a ZEV standard, 35% of new vehicle sales would be ZEVs, this says that 35% of buyers of new vehicles voluntarily choose ZEVs across the whole United States (or whatever region the EPS is modeling, but let’s use the U.S. as our example in this email).
It would be very unlikely if all of those buyers were concentrated in particular U.S. states. Most likely, the 35% of total buyers who want ZEVs are spread out across the country, with at least a few in each of the 50 states. Intuitively we know there may be greater concentrations in some areas than others (such as cities vs. rural areas), but we don’t model that sub-regional detail. In essence, the model assumes there is one region, so it doesn’t matter how people are spread out throughout that region, and if the natural 35% sales is lower than the ZEV requirement, it reallocates to meet the national ZEV requirement. There would be no justification for exceeding it because the natural tendency of the buyers in this example was to not meet the requirement.
If we assume the 35% entirely comes from sub-regions meeting sub-regional targets, and we allocate all the remainder, then we would overstate ZEV sales, because some of the 35% we initially allocated actually consisted of buyers in regions that are not meeting their targets, like some buyers who choose EVs even though their sub-region has no EV requirement.
So our current method works well for single, homogenous regions without sub-regional policies. That might be good when modeling a single U.S. state. But when modeling the entire U.S., where there can be policies at sub-regional scale, you are correctly pointing out that we might be under-estimating ZEV deployment. However, I think your proposed fix goes too far in the opposite direction and would overstate ZEV deployment. We should try to find something in the middle.
If we aren’t doing too many math operations at sub-regional scale, we could introduce a subscript for sub-region, each of which can have its own ZEV percentage requirement. Then we keep the current approach (which first allocates based on pure consumer preferences, then shifts sales if necessary to meet the target), but do it separately in each sub-region, so that a sub-region with a 100% ZEV mandate like California would be properly represented, without affecting the number of EV buyers in, say, South Carolina. I think that’s feasible here, since the only thing varying at sub-regional scale is the ZEV mandate, so we just do the shift 50 times instead of 1 time, which should be quick enough not to have any meaningful runtime impact.
From Robbie:
If I understand correctly, I think we are essentially converging on the same thing, but (a thought) rather than have multiple regions, we might be able to do it with just two regions: states with and without ZEV rules. If we can do it this way, we don’t need to add a subscript. We would do two passes, first one for states with ZEV rules and then one without. For the states with ZEV rules, we keep the current approach where we allow a cost-based pass, then adjust as needed to comply with the target. For the second pass for remaining states, we just let the model do a normal cost pass.
The example I gave in the spreadsheet is functionally equivalent, I think, because it looked at case where all the states with ZEV standards had a 100% target. But as soon as that’s less than 100%, my approach would overcompensate, to your point.
To sum, I think your approach is right, but I wonder if we can just use percentages of coverage of sales rather than trying to add a new subscript for multiple regions?
From me:
If every state has its own unique percentage, in theory, we’d want one bucket for every unique percentage (though each percentage could be shared by multiple states). But if it’s a bunch of states with 100% requirements and a bunch of other states with 0% requirements, then two buckets is enough.
Rather than conceive of buckets as sub-regions, we could conceive of them as ZEV stringency tiers. The simplest approach has just a 0% and a 100% tier, but if you want more detail, we could do something like eleven tiers (0%, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, 100%) and put each sub-region (U.S. state) into the closest tier. The nice thing about that is that it scales up to any number of sub-regions without impacting model run speed, because you’re summing them into eleven tiers in Excel before you run calculations in Vensim. And with the tiers only separated by 10% gaps, any loss in accuracy would be imperceptible.
That might be helpful in modeling a transition toward 100% EV mandates, where there will be intermediate percentages in various years, as well as the possibility of some purple states enacting some ZEV policies but stalling out before they reach 100%.
From Megan:
Thanks for sharing this, Robbie. I like the idea of this kind of update, and Jeff’s approach makes sense to me as a way of capturing variation without having to add an unmanageable number of subscripts.
One other point to consider is that our approach currently sets a standard for battery electric vehicles only, but CA’s standard also applies to plug-in hybrids (and I believe hydrogen vehicles, which will be a much smaller share). If we’re implementing this change, I think we’d likely want to add the ability to specify which vehicle technologies contribute to meeting the standard when calculating the first pass.
Hey, @robbieorvis, quick question on the implementation of a policy lever for ZEV standards after dividing vehicle sales into stringency tiers.
Suppose for the U.S. national model we have 50 states with 50 different ZEV percentage requirements in a given year, and we group them into tiers like 0%, 10%, 20%... 90%, 100%. Then the user specifies a ZEV standard in that year of 40%.
Do you want me to pull up all the states that are below 40% so that their standards are 40% without changing the standards of the states that are already exceeding 40% in that year? This is the most straightforward way. But it would over-comply with the standard on a nationwide average basis.
Or do you want me to take a weighted average of the car sales in the eleven stringency buckets (suppose that average comes out to 33%), then decide how to allocate increases so that a nationally weighted average of 40% is achieved? It's not obvious how to allocate these increases. One option would be to allocate the required increase to all states in proportion to how far their current requirement is from 100%, but this might be seen as unfair because states that have a goal of 80% in that year might argue that they shouldn't need to further increase their target. Perhaps a better option is to allocate the increase to all states that are below 40% in proportion to how far their targets are from 40% in that year. This is similar to the first option but much more complex to calculate because you're only increasing their targets enough to reach a nationally weighted average of 40% rather than increasing all the laggard states' targets to 40%.
Note that we'd actually be working with the eleven stringency buckets and the amount of cars sold that are in each bucket, not actually 50 states, but I used "states" in the description above to make it more readable. It should be understood to actually refer to buckets of states.
Also note that the latter approach would require complicated new input data that says how many vehicles are sold in each stringency bucket in each year (which can change as states move between buckets) in order to perform the weighting, and BAU data for that variable would not necessarily remain valid in policy scenarios (because states would move between buckets in different years than in the BAU dataset), so we probably would need to actually track 50 states rather than 11 stringency buckets if we wanted to perform the weighting accurately. Which, I suppose, raises the question of why we'd use stringency buckets at all. Maybe we just introduce a sub-region subscript with 50 elements and work with that.
Given the greater complexity and input data requirements of handling the nationally weighted average, it would be a whole lot easier if we could define the ZEV standard policy lever as the minimum required ZEV sales percentage in each state, rather than the minimum required nationally weighted average ZEV sales percentage. Is that okay?
Hi Jeff, I’m not sure I fully follow, but here’s what I’m thinking. I just had to do this in Excel, replicating the logit functions, for some other modeling with some external partners. I will upload a copy of that file for you to see.
In the spreadsheet modeling I did, my approach was:
That is all just to get BAU values. Now, I think your question also pertains to how to handle situations where a policy comes into place on top of this. My feeling is that if the ZEV standard set by the user is less than the values calculated above, it is non-binding. Because the model is single region and we are looking at national policy, I think it should drive additional adoption on top of the sum of the state policies. I believe that is the easier approach, but not entirely sure based on your prior comment. It’s also basically the same approach to the RPS, where a setting that is too low fails to drive incremental adoption.
I hope that answers your question, but let me know if not. I will upload the file I am talking about for you to review as well.
Here is the file I mentioned 20221116_ICCT IRA Modeling v4.xlsx
Note also: this approach moves the complexity IMO into the input data and out of Vensim. All we need are data on which vehicles would qualify (if you want to use a ZEV instead of just the EV standard approach; if not then you don't even need this because you don't need the first allocation pass... but in the long run we need to move to this type of approach because of how vehicle standards are designed), and the percentages for the scope of sales covered by ZEV requirements and the share of sales in states with ZEV standards that are not ZEV. subtracting these two from 1 give you the remaining value for states not covered by ZEV programs.
@robbieorvis - thanks for going over the BAU approach and uploading the Excel file.
The idea of doing two passes, a ZEV-required pass and an unrestricted pass (for the remaining newly sold vehicles), similar to the RPS-qualifying and unrestricted passes in the electricity sector, makes sense to me. I can implement it that way. I don't think we need three passes because any states with no legislative ZEV requirement can have 0% of their demand in the ZEV pass and 100% of their demand in the unrestricted pass.
There are really only two wrinkles to iron out:
When the user has set a national required ZEV percentage that is higher than the BAU percentage, which states' required ZEV percentages do we increase, and by how much? This is an issue of mapping a national policy lever onto policy that is modeled at the 50-state level. Do we allocate the required increase to all states, in proportion to their distance from 100% market share? Do we allocate them only to the states that are currently under 40%, in proportion to their distance from 40% market share? I will assume the latter is what we want, unless I hear otherwise. (We then use the adjusted ZEV requirement percentages in the policy case's two allocation passes as detailed above.)
This approach requires a subscript with elements for all 50 states, which was something we thought we might avoid by using stringency tiers, which would generalize better to any country or region in the world that contains sub-regions with varying ZEV policies. But if you want to avoid over-complying with the standard on a national level, we need to find the weighted average ZEV requirement, which requires state-level sales data to use as weights. We can map the state-level sales data onto stringency tiers, but we run into problems when states move between stringency tiers, because that changes the weights of the stringency tiers. We can't track how the tiers' weights change unless we are tracking which states are in which tier, which means we'd need a 50-state "Subregion" subscript anyway. So in the end, using stringency tiers gives us nothing, and we ought to be allocating for 50 states directly to avoid unnecessary and unhelpful complexity. (It will also somewhat increase accuracy because we won't have to quantize states' ZEV requirements into specific tier buckets.) So it's probably better to work with a subregion subscript after all.
The Excel sheet you uploaded is more complex (and more specific to the idiosyncrasies of the Inflation Reduction Act) than we would want for core structure in the EPS. We can't move stuff into Excel unless the results are not affected by the policy levers, or we're making a parameter like deaths-per-unit-emissions that we use in the EPS. Fortunately, 50 allocations shouldn't be bad. Logit functions are fast because they are single-pass (unlike ALLOCATE AVAILABLE), and also, 50 allocations is a lot less than we get in the electricity sector if we try to do one for every hour of the year (8760). So I'm not worried about runtime impacts in this case.
I think I know what I need to do to move ahead with this using a new "subregion" subscript with 50 state elements (or maybe up to 56 elements if we have vehicle sales and ZEV standard data for Washington D.C. and for some or all of the five inhabited U.S. territories).
Hi Jeff,
Thanks for looking at this. Sorry that Excel doc is confusing. It is indeed trying to do more than we want (and to cover the IRA). The main area to look is the ACCII tab and the first two passes for the passenger vehicle allocations.
A few responses/comments:
Regarding 2 vs 3 passes: I don't see how we get away from three passes. If you just put the non ZEV share (ZEV states share of sales not covered + states with ZEVs) you will get some of that allocation to go to ZEVs, which would mean the states are overcomplying on average. The only way to avoid this that I can see is to do three passes: one for ZEVs only, one for non-ZEVs only, and one for all vehicles, then apply the percentages to each of those based on the sum of states with ZEV programs and the ZEV program targets.
I'm not clear on why we need a state subscript, although perhaps it's related to three passes instead of two. In theory, with three allocations and percentages covered by each allocation, we should be covered in the BAU case. If we think about a national policy, it's not that the national policy is applied in all the states (which would require a state subscript), it's that a national standard allows for different compliance across the states, so long as the national total complies. In that scenario, if the national standard is weaker than the sum of the state standards, it would simply be non-binding, just like the RPS policy. Policy values would need to be higher than the sum of the states to drive any additional uptake. We could calculate this by comparing the policy setting to the BAU achieved ZEV shares and computing the incremental amount of sales that need a ZEV allocation. The area I'm a little uncertain of is how the three allocation passes would fit together for a national standard in the policy case, but one option would be to do a single allocation with all the policies in place in the non-BAU calculations, and then take the max of either the BAU or the policy calculations. That should only drive incremental ZEV deployment when the combination of national policies is stronger than the combination of state policies in the BAU. It's a little weird though.
We would also need a boolean input data for qualifying vehicle types, if we want.
One potential thought: we could change the input data to be: share of national sales covered by ZEV and weighted average ZEV target. I think that would give us everything needed to calculate the percentages for all three passes (sum of state sales ZEV amount, sum of state sales (1-ZEV amount), and share of total national sales not covered by ZEV.
Okay, I think I understand your thinking about the three passes now. Your vision for the second pass is to not allow any ZEVs to be sold in that pass, to prevent over-compliance at the state level. That is different from the electricity sector's RPS, which has only two passes and allows more wind or solar to be built in the unrestricted (second) pass, if renewables are the most economical option.
I agree with you that two passes like the RPS at national scale doesn't work well here because, as you say, it would lead to systematic over-compliance nationally. Got it. 😊
However, I think it is problematic to not allow any individual state to exceed its ZEV mandate in ZEV sales percentage. There could be a state, province, country, or region that aims to use other policies, like subsidies, to boost EV sales with only a weak mandate as a floor, so allowing over-compliance would be critical. Even in the U.S., there can be over-compliance in states when standards are still weak, such as in early years while a state standard is just beginning to phase in. For example, if a state right now has no ZEV mandate but institutes one in the future, that state will be in over-compliance in the early years of their mandate because the state will already have some (perhaps many) EVs on the roads.
Suppose a state implements a ZEV mandate of 0.01% that stays constant through 2050. With the three-pass approach above, that state would have near-zero EV sales through 2050 because all of the sales would be funneled into the second pass, which disallows EVs.
So, I think we shouldn't have a "special case" for states with no standards that is handled differently from states with weak standards. To avoid bugs and problems with edge cases and early years of policy phase-in, we should have a methodology that operates in a smooth, continuous way for states that range from no EV standard all the way up to a 100% EV standard. I think we should avoid over-compliance only in cases where it is not economical to over-comply.
The current approach allocates everything freely, then increases the share of ZEVs if the free allocation didn't meet the standard. I think this is still broadly correct and does not overcomply. As I understood it, the issue with the current approach is whether, when done nationally, it might understate ZEV deployment because it allows states that are over-complying with their ZEV standards (or that have ZEVs but have no ZEV standard) to "make up for" some states that are actually missing their ZEV standards. If we model every state separately, we can ensure no state is missing its ZEV standard. Is that still the core issue?
Then, as a separate matter, we could allow multiple vehicle technologies to qualify for a ZEV standard, set by a flag, similar to RPS-qualifying technologies. We allocate amongst qualifying vehicle types when determining which type of ZEVs to add when we need to increase ZEVs in a state to meet that state's target.
So in summary:
I see no way around doing this with a "subregion" subscript at this point because, as I understand it, the core inaccuracy we want to address is the way that states that are over-complying with their ZEV standards (or that have ZEVs but have no ZEV standard) can "make up for" some states that are actually missing their ZEV standards. I don't see how to get that state-level resolution without a sub-region subscript.
If you think this situation is so rare that it never happens or doesn't matter, then I think the existing structure is already accurate, except for not yet including the customizable definition of which vehicle technologies qualify as ZEVs.
Had one more thought: we might be able to solve overcompliance by just using a max function of the logit results. If the third allocation produces shares greater than the sum of the first two, then we just use the third pass for the states with ZEV.
Sent from my iPhone
On Nov 17, 2022, at 9:11 PM, Jeff Rissman @.***> wrote:
Okay, I think I understand your thinking about the three passes now. Your vision for the second pass is to not allow any ZEVs to be sold in that pass, to prevent over-compliance at the state level. That is different from the electricity sector's RPS, which has only two passes and allows more wind or solar to be built in the unrestricted (second) pass, if renewables are the most economical option.
I think it is problematic to not allow any individual state to exceed its ZEV mandate in ZEV sales percentage. There could be a state, province, country, or region that aims to use other policies, like subsidies, to boost EV sales with only a weak mandate as a floor, so allowing over-compliance would be critical. Even in the U.S., there can be over-compliance in states when standards are still weak, such as in early years while a state standard is just beginning to phase in. For example, if a state right now has no ZEV mandate but institutes one in the future, that state will be in over-compliance in the early years of their mandate because the state will already have some (perhaps many) EVs on the roads.
Suppose a state implements a ZEV mandate of 0.01% that stays constant through 2050. With the three-pass approach above, that state would have near-zero EV sales through 2050 because all of the sales would be funneled into the second pass, which disallows EVs.
So, I think we need to allow over-compliance even in U.S. states with standards. We shouldn't have a "special case" for states with no standards that is handled differently from states with weak standards. To avoid bugs and problems with edge cases and early years of policy phase-in, we should have a methodology that operates in a smooth, continuous way for states that range from no EV standard all the way up to a 100% EV standard. I think we should avoid over-compliance only in cases where it is not economical to over-comply, on the basis of the underlying economics and other policies like an EV subsidy, then over-compliance is the correct result.
The current approach allocates everything freely, then increases the share of ZEVs if it didn't meet the standard. I think this is still broadly correct and does not overcomply. As I understood it, the issue with the current approach is whether, when done nationally, it might understate ZEV deployment because it allows states that are over-complying with their ZEV standards (or that have ZEVs but have no ZEV standard) to "make up for" some states that are actually missing their ZEV standards. If we model every state separately, we can ensure no state is missing its ZEV standard. Is that still the core issue?
Then, as a separate matter, we could allow multiple vehicle technologies to qualify for a ZEV standard, set by a flag, similar to RPS-qualifying technologies. We allocate amongst qualifying vehicle types when determining which type of ZEVs to add when we need to increase ZEVs in a state to meet that state's target.
So in summary:
I see no way around doing this with a "subregion" subscript at this point because, as I understand it, the core inaccuracy we want to address is the way that states that are over-complying with their ZEV standards (or that have ZEVs but have no ZEV standard) can "make up for" some states that are actually missing their ZEV standards. I don't see how to get that state-level resolution without a sub-region subscript.
If you think this situation is so rare that it never happens or doesn't matter, then the existing structure is already accurate, except for not yet including the customizable definition of which vehicle technologies qualify as ZEVs.
— Reply to this email directly, view it on GitHubhttps://github.com/EnergyInnovation/eps-us/issues/256#issuecomment-1319456924, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AK5N6SMHB6ZPWZZWTTYBT4LWI3QVXANCNFSM6AAAAAARRO3SO4. You are receiving this because you were mentioned.Message ID: @.***>
Yes, that seems like a reasonably clean way to structure the code flow. It's probably a bit cleaner than the approach outlined in my bulleted summary above! 😊
But I think it still has to be done at the subregion (state) level if we want to prevent the issue where some states can miss their targets if other states over-comply.
Check out this example I put together, attached. I think the situation you are describe is avoided by using the MAX approach:
1) If we use the MAX system I described, then ZEV-standard states will always have at least as high of a sales share as the non-ZEV states, because whenever a national ZEV standard or incentives cause greater shares in the allocation than the the ZEV standard, the allocation will use the greater of the two.
2) There cannot be a case where, due to other states over-complying, ZEV states can undercomply, because of the dynamic above, unless I am misinterpreting your point or failed to calculate something.
Maybe I'm just not seeing it, but based on the three region example I just put together, it looks like we won't need a region subscript the single- and multi-region values are identical. ZEV Example.xlsx
One other thing I learned this morning. We may indeed only need 1 allocation, because the ratios are the same between technologies across the different allocations I mentioned. For example, even if you have an allocation across all technologies that gets you 20% BEV, 5% PHEV, and 75% ICE, if you just did an allocation with the ZEV qualifying BEVs and PHEVs you would get 80% BEV and 20% PHEV (a ratio of 4:1). So we can just do a single allocation and scale the results to the qualifying technologies, which makes things simpler than have to program multiple allocations.
Updated file for your review ZEV Example.xlsx
Completed in 17ea4c6. As we discussed today, we're using sub-regions now, not stringency buckets nor a nationwide weighted average. The code is very clean and should be largely self-explanatory when examined. The EV minimum and hydrogen vehicle minimum policies were replaced with the new ZEV standard policy. I am using real data I found, not stand-in data, but nonetheless someone should look over trans/BRZSPbS
just to ensure accuracy before releasing.
This entailed a lot of changes to a lot of different files. It needs its own release and must not be included in 3.4.3. This new sub-region feature is irrelevant to the U.S. state models because U.S. states don't have sub-regions with their own, varying ZEV standards. Trying to release them on this platform would be a huge amount of work to update 40+ files in each U.S. state repo for zero benefit in terms of model capabilities. Therefore, I've tagged the release prior to commit 17ea4c6 as 3.4.3 and would like you to release that tagged commit as 3.4.3 (updating the Version History and Download links in the docs accordingly). This EVs issue only pertains to the U.S. national model and can be released as a 3.4.4 release, which will not be the platform used by the U.S. state models.
Note that because the new ZEV standard only applies to road vehicles, when we removed the EV and H2 vehicle minimum policies, we lost the ability to require new non-road vehicles to be electric or to use hydrogen, such as rail electrification or hydrogen ships. We might consider putting back some policies mandating fuel shifting for non-road vehicles when we do a larger transportation overhaul. But that should not hold up 3.4.4, and should not come before we finish the electricity sector updates in 3.5.
In fact, this really must be the last feature we put in the 3.4 series because it is getting to be far too burdensome to port features into 3.5.
From Robbie: