EnergyInnovation / eps-us

Energy Policy Simulator - United States
GNU General Public License v3.0
22 stars 7 forks source link

Calculate the domestic content share of consumption for ISIC 05T06 in Vensim to account for differences in coal vs. crude oil #130

Closed mkmahajan closed 3 years ago

mkmahajan commented 3 years ago

India has significant coal production, but little crude oil production. But coal and crude production are both lumped under ISIC 05T06. In a scenario with lower transportation fuel use, for example, you see a substantial negative change in ISIC 05T06 output, which should be pretty much entirely due to changes in crude demand. That change in output gets multiplied by a DCSoCbIC value of ~0.45, based on OECD data. But looking at our data from BFPIaE, the domestic content share of crude oil consumption specifically is more like 10-15%.

Could we calculate the domestic content of each fuel type using BFPIaE, then use that to swap in a calculated variable for the ISIC 05T06 domestic content share of consumption? We could weight the domestic content of each fuel type by the change in revenue associated with that fuel.

If this makes sense and is straightforward, could we add this to the 3.1.1 update list?

jrissman commented 3 years ago

The issue you raise here makes sense, but I'm not a fan of the proposed fix. It's a hack or work-around, and the more of those you put into the code, the messier it becomes, which has negative implications for keeping the code maintainable and understandable, and can lead to bugs. (For instance, it can be difficult to know if changing this one variable for this ISIC code would lead to invalid results elsewhere, since there are many places in the code that use ISIC codes, and they were programmed with a particular assumption about what each ISIC code means.) I've tried to be very meticulous over the years to minimize one-off hacks, which is why the simulator is able to be as complex as it is today without being a nightmare to debug or understand.

The core need here is simply an ISIC code split. According to this page of ISIC code definitions, ISIC 05 is "mining of coal and lignite" and ISIC 06 is "extraction of crude petroleum and natural gas". The only reason they are together is because the OECD grouped them in its IO tables. Splitting them up would improve accuracy not only for DCSoCbIC, but for every variable that contains data for ISIC 05-06. So it seems to me like the cleaner way to do this, which won't violate any assumptions elsewhere in the I/O model, would be to split up "ISIC 05-06" into ISIC 05 and ISIC 06. This would be just like when we split up the OECD's grouping of ISIC 20-21 into chemicals (ISIC 20) and pharmaceuticals (ISIC 21), because it was not giving us realistic results when they were grouped.

My memory of splitting up ISIC 20-21 is that it was not an easy task and took me a week of work. Having had that experience already, it would be faster to split up ISIC 05-06, but it won't be super quick. I do not think we should put this into 3.1.1, which is meant to be a very small, incremental update that is released ASAP and that does not involve any changes to input data requirements relative to EPS 3.1.0 (other than FoPITY for the new subscripting of policy levers). But if this issue is more important than the other things we green-lit for 3.2, we might consider doing it alone and releasing it as EPS 3.1.2.

I am going to need to be spending most of my time on ISD over the next year, so we're going to need to be careful about what it's worth doing structurally and what the EPS team can live without.

robbieorvis commented 3 years ago

Hi Jeff,

Understand your concerns here and the optimal path forward. I’m wondering if this is an interim fix we can put in, since it is a simple few variable updates, that we replace down the road with the proper fix given time limitations.

The issue here is that in India, one of the main questions we are being asked in our research is how the coal mining industry will be affected (for a concrete example, we are working with an influential member of parliament who wants to look at deep mid-century decarbonization scenarios, but is highly interested in the coal mining sector impacts as part of the report focus). Since vehicle electrification and other oil/gas reducing policies are part of the broader decarbonization picture, we are worried that the results for the coal mining industry will be significantly wrong with the current approach and will significantly overstate the job losses. That’s of course not a message we want to send. It just so happens that in this instance the questions that are being asked directly relate to the need to break out ISIC05 and ISIC06.

Given that context, would you be open to implementing this as a short term fix, with the goal in the long run to split the codes (or more broadly find a different set of international IO data with more detail)?


Robbie Orvis Director of Energy Policy Design Phone: 415-799-2171 98 Battery Street, Suite 202 San Francisco, CA 94111 www.energyinnovation.orghttp://www.energyinnovation.org/ [cid:image001.jpg@01D6D396.872089A0]


Check out our new book, Designing Climate Solutions: A Policy Guide for Low-Carbon Energyhttps://www.amazon.com/Designing-Climate-Solutions-Policy-Low-Carbon/dp/1610919564 Available wherever books are sold

[Policy Design book cover]

From: Jeff Rissman notifications@github.com Sent: Tuesday, December 15, 2020 9:22 PM To: Energy-Innovation/eps-us eps-us@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: Re: [Energy-Innovation/eps-us] Calculate the domestic content share of consumption for ISIC 05T06 in Vensim to account for differences in coal vs. crude oil (#130)

The issue you raise here makes sense, but I'm not a fan of the proposed fix. It's a hack or work-around, and the more of those you put into the code, the messier it becomes, which has negative implications for keeping the code maintainable and understandable, and can lead to bugs. (For instance, it can be difficult to know if changing this one variable for this ISIC code would lead to invalid results elsewhere, since there are many places in the code that use ISIC codes, and they were programmed with a particular assumption about what each ISIC code means.) I've tried to be very meticulous over the years to minimize one-off hacks, which is why the simulator is able to be as complex as it is today without being a nightmare to debug or understand.

The core need here is simply an ISIC code split. According to this page of ISIC code definitionshttps://siccode.com/isic-code-hierarchy/b/mining-quarrying, ISIC 05 is "mining of coal and lignite" and ISIC 06 is "extraction of crude petroleum and natural gas". The only reason they are together is because the OECD grouped them in its IO tables. Splitting them up would improve accuracy not only for DCSoCbIC, but for every variable that contains data for ISIC 05-06. So it seems to me like the cleaner way to do this, which won't violate any assumptions elsewhere in the I/O model, would be to split up "ISIC 05-06" into ISIC 05 and ISIC 06. This would be just like when we split up the OECD's grouping of ISIC 20-21 into chemicals (ISIC 20) and pharmaceuticals (ISIC 21), because it was not giving us realistic results when they were grouped.

My memory of splitting up ISIC 20-21 is that it was not an easy task and took me a week of work. Having had that experience already, it would be faster to split up ISIC 05-06, but it won't be super quick. I do not think we should put this into 3.1.1, which is meant to be a very small, incremental update that is released ASAP and that does not involve any changes to input data requirements relative to EPS 3.1.0 (other than FoPITY for the new subscripting of policy levers). But if this issue is more important than the other things we green-lit for 3.2, we might consider doing it alone and releasing it as EPS 3.1.2.

I am going to need to be spending most of my time on ISD over the next year, so we're going to need to be careful about what it's worth doing structurally and what the EPS team can live without.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/Energy-Innovation/eps-us/issues/130#issuecomment-745720370, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AK5N6SMXJ2DQMOWI2LIT55DSVAKTNANCNFSM4U5GEWYA.

jrissman commented 3 years ago

That concern makes sense. Let's talk about it on the call today. Maybe we can do something temporary, if it doesn't cause problems for the U.S. national model. Or maybe I can try to implement this one ISIC code split as quickly as possible, since we are an energy policy model, and getting better resolution on coal vs. oil and gas is pretty central to what we do.

I suspect that moving to a completely different set of ISIC codes from a different international source isn't really on the table for 2021, but you could pull from that source to populate ISIC 05 and ISIC 06, like we will from BLS/BEA for the U.S.

robbieorvis commented 3 years ago

Sorry we didn’t’ get to this today.

We’re pressing forward with a China update as well, and there too there is a lot more coal production than oil and gas, so I can see a similar issue arising there. So if possible to do this hotfix, it would be great. I am happy to connect tomorrow or Friday to continue the discussion, if you’d like.


Robbie Orvis Director of Energy Policy Design Phone: 415-799-2171 98 Battery Street, Suite 202 San Francisco, CA 94111 www.energyinnovation.orghttp://www.energyinnovation.org/ [cid:image001.jpg@01D6D3E4.344F8840]


Check out our new book, Designing Climate Solutions: A Policy Guide for Low-Carbon Energyhttps://www.amazon.com/Designing-Climate-Solutions-Policy-Low-Carbon/dp/1610919564 Available wherever books are sold

[Policy Design book cover]

From: Jeff Rissman notifications@github.com Sent: Wednesday, December 16, 2020 1:40 PM To: Energy-Innovation/eps-us eps-us@noreply.github.com Cc: Robbie Orvis robbie@energyinnovation.org; Comment comment@noreply.github.com Subject: Re: [Energy-Innovation/eps-us] Calculate the domestic content share of consumption for ISIC 05T06 in Vensim to account for differences in coal vs. crude oil (#130)

That concern makes sense. Let's talk about it on the call today. Maybe we can do something temporary, if it doesn't cause problems for the U.S. national model. Or maybe I can try to implement this one ISIC code split as quickly as possible, since we are an energy policy model, and getting better resolution on coal vs. oil and gas is pretty central to what we do.

I suspect that moving to a completely different set of ISIC codes from a different international source isn't really on the table for 2021, but you could pull from that source to populate ISIC 05 and ISIC 06, like we will from BLS/BEA for the U.S.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/Energy-Innovation/eps-us/issues/130#issuecomment-746800972, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AK5N6SPFZ4XC7U4M5A2Q3TTSVD5JNANCNFSM4U5GEWYA.

jrissman commented 3 years ago

I talked with Megan and Olivia about it after you left the call. I'll be trying to squeeze this in soon. We didn't make a determination on whether to use the quick method or the proper method - I'll probably do it using the proper method, even though it takes longer.

mkmahajan commented 3 years ago

One thing we discussed after you dropped off the call is whether we can find an international source that breaks out ISIC 05 and ISIC 06. The World Input Output Database I used for the chem/pharma split in India does not break this out. @robbieorvis I remember you had looked at a few international sources previously so was planning to check in with you on those and do a scan to see if I could find a good source.

robbieorvis commented 3 years ago

I’ve not come across a database that breaks this apart, and if it did, I think it would probably follow something similar to the first iteration we outlined.

In other words: for splitting out output, it would just take the value of coal products and the value of oil+gas products, determine a ratio, and multiply by the default data in ISIC 05t06 to allocate. This is something we could do endogenously in the EPS.

The shares of value added and labor we could keep the same (seems sort of reasonable to me), or we could find this data, at least for the US, from BLS. There are not a lot of countries that have BOTH large coal mining and natural gas/oil extraction. The US is kind of an outlier there, which is interesting. So keeping the same assumptions as a general rule and using custom data for the US model in particular if we want would be fine with me.

The trickier bits would be with the IO table and the DLIM table. For the IO table, because we actually have energy demand by industry and know the types of energy those industries are using, this again could be done endogenously in the EPS. We also have energy imports calculated and domestic energy production calculated, so it seems feasible to do this endogenously for DLIM as well.

Jeff is much savvier than I am on this, so I will defer to him, but part of the reason I had proposed the kind of quick fix we mentioned is that I think we have most of the data in the model to do this split endogenously (though admittedly our first suggested fix was much cruder – pardon the pun – and simpler, but not really correct methodologically).


Robbie Orvis Director of Energy Policy Design Phone: 415-799-2171 98 Battery Street, Suite 202 San Francisco, CA 94111 www.energyinnovation.orghttp://www.energyinnovation.org/ [cid:image001.jpg@01D6D45E.A696E6D0]


Check out our new book, Designing Climate Solutions: A Policy Guide for Low-Carbon Energyhttps://www.amazon.com/Designing-Climate-Solutions-Policy-Low-Carbon/dp/1610919564 Available wherever books are sold

[Policy Design book cover]

From: mkmahajan notifications@github.com Sent: Wednesday, December 16, 2020 8:09 PM To: Energy-Innovation/eps-us eps-us@noreply.github.com Cc: Robbie Orvis robbie@energyinnovation.org; Mention mention@noreply.github.com Subject: Re: [Energy-Innovation/eps-us] Calculate the domestic content share of consumption for ISIC 05T06 in Vensim to account for differences in coal vs. crude oil (#130)

One thing we discussed after you dropped off the call is whether we can find an international source that breaks out ISIC 05 and ISIC 06. The World Input Output Database I used for the chem/pharma split in India does not break this out. @robbieorvishttps://github.com/robbieorvis I remember you had looked at a few international sources previously so was planning to check in with you on those and do a scan to see if I could find a good source.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/Energy-Innovation/eps-us/issues/130#issuecomment-747138602, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AK5N6SKJTR4XSV3KC4S5JFDSVFKY5ANCNFSM4U5GEWYA.

mkmahajan commented 3 years ago

I followed up with Deepthi at WRI about timeline and while I thought there may be further delays to get stakeholder feedback into the model, they're forging ahead and would like to finalize things in early January to have final numbers for their report. Jeff, do you think it's feasible to get the proper method in shortly after we get back in the new year, or does this suggest we should do a quicker fix?

jrissman commented 3 years ago

Okay, I will look into this tonight and possibly put a completed, temporary fix into place, or else get started on the full fix.

jrissman commented 3 years ago

Since the EPS team seemed to really need this item completed ASAP, and I didn't want to put in a hacky temporary fix, I went ahead and split ISIC 05T06 into coal mining (ISIC 05) and oil and gas extraction (ISIC 06) tonight. It turned out to only require about eight work hours, much faster than the week it took to split chemicals and pharma. That's because I'd already carefully set up the Excel files to facilitate this kind of splitting when I did the chem/pharma split, and that earlier work was still fresh in my memory, so I could move quickly with no wrong turns.

Commit fb7e384 contains changes to 50 separate files, 47 of which are input data files. You can use the file list in this commit to help tell our India partners which files to update (or to help you make the updates on their behalf).

U.S. data come from BLS (employment) and BEA (everything else). For some of the variables subscripted by ISIC code, the values for ISIC 05 and ISIC 06 are very different. For example, ISIC 06 (oil and gas) is much bigger and has large imports, whereas ISIC 05 (coal) is small and has almost no imports. Due to differences like these, having properly split these ISIC codes will likely greatly improve accuracy in the U.S. simulator, on top of any benefits it provides for the India EPS.

This could use some more testing, either by someone who is working Mon-Wed this coming week, or in January when we are all back. But because it appears to be working and I think it's done, I'm going to close this issue.