Allocation of fuel industry revenue by ISIC code producing odd results

[mkmahajan]

While testing the new ISIC 05/ISIC 06 split, I found some odd behavior. I tested a scenario with 100% LDV and HDV EV mandates and using FoPITY-2 (e.g. reaching 100% electric LDV sales by 2035). This results in a large negative spike in GDP in 2036. The spike is originating when we allocate the change in ngps revenue to different fuel types (the variable Direct Change in Fuel Industry Revenue by ISIC Code), even though the trends in Change in Fuel Industry Revenue by Fuel are smooth.

I wasn't able to identify a specific problem with the current method, but it appears to cause unreasonable results in this example.

One other note here is that the formula for Direct Change in Fuel Industry Revenue by ISIC Code adds the nuclear share of the 'other energy suppliers' change in revenue into ISIC 05. But uranium/thorium mining falls under ISIC 0721, so I believe this should be part of our ISIC 07T08 grouping, which covers metal ores.

[jrissman]

Thanks for this testing. I'll look into it on Thursday.

[jrissman]

This is happening because the change in revenue for each of the NGPS component fuel categories (natural gas, crude oil, and refined petroleum products) is very large (in either the positive or the negative direction), while the total change for the NGPS fuels is close to zero, with a little bit of bumpy noise (because the changes in the component fuels are almost cancelling each other out). When we divide these very large changes for each fuel type by a very tiny total number, it magnifies the noise in the "total" line so that they become huge spikes in the fuel-specific lines. We probably need to be using addition and subtraction here, not multiplication and division, to avoid amplifying any noise. I'm going to work on this now.

[jrissman]

Commit 81a9b39 completely fixes this issue. We no longer use the by-fuel totals to divide up certain fuel supplier cash flow entities' cash flows. Instead, we use the by-fuel totals directly in the IO model, no longer relying on the "fuel supplier cash flow entities." This avoids the intermediate step of division that introduced so much noise. Here are before and after comparison graphs:

Before:

After:

There is now a minor discrepancy (picked up on the debugging sheet) between the cash flows for the sum of those five entities and the total of all revenues when divided up by fuel. Perhaps something very minor is missing from either the cash flow entities or the fuel-specific totals. I can look into it now.

The thing is, we aren't using the five fuel-supplier cash flow entities for very much any more, except in output graphs. It would probably be cleaner and better for the long-term maintainability and health of the code to remove the five fuel-supplying cash flow entities, and just use the division of revenue by fuel that we now have, instead of divvying up this change in revenue into two sets of differently-categorized buckets, and use those different buckets in different places in the model. Dividing it up by fuel is more precise than dividing it up by fuel-supplying cash flow entity, so we would want to keep the more precise set of buckets and remove the five fuel-supplier cash flow entities. This would require some amount of meticulous work to update various variables and output graphs to use the fuel-specific revenue totals rather than the fuel-supplying cash flow entities. And it requires updating a variety of input data tables to remove the five fuel-supplier cash flow columns or rows (which often contain zeroes anyway). In cases where they might not contain zeroes, we'd need to see where to put those data, or if they can be disregarded or grouped with the other industries. I'll look into this today also.

In the meanwhile, the latest version now in the "develop" branch fixes the two issues you raised, and it is an improvement over what we had before, so you can start using it right away for in-development EPS regions.

[mkmahajan]

This is great, thank you Jeff!

[jrissman]

Commit e98c556 fixes the discrepancy mentioned in the second paragraph of my last comment in this thread. Please update to the latest version of EPS.mdl and EPS.vpmx in the "develop" branch to include this fix.

This commit narrowly fixes the bug noted above. It does not attempt to make any changes to the composition of the "cash flow entity" subscript, which is a larger task.

[jrissman]

I found and fixed a couple more obscure missing subsidies/rebates in the fuel-specific totals in commit 6a31c8d. I think this is done and fully captures all relevant revenues being paid to fuel suppliers.

Even these relatively targeted bug fixes apparently took about 6-7 hours, and to try to overhaul the Cash Flow Entity subscript would take much longer. There's no need to do that right now, as there is no flaw in the current model (no problem with the calculations or outputs) - it's just a refactoring thing. I'll make a separate, lower-priority issue for it and close this issue.