Closed orichters closed 1 year ago
Changes of this PR seem to be mostly energy system related. It would be surprising that this should cause such macro change. My suspect would be the change of the CES parameters (probably derived from a new calibration which itself could be include a number of different changes from previous PRs). Also the inputRevision number changes. Could there be an issue with new input data?
@mleimbach @orichters How are the 2005 non-ESM capital stocks determined in REMIND? are they an input data to which the variable is fixed?
I have to remind myself. So without guarantee: capital stocks are free, but more or less predetermined by the calibration which provides the (share and) efficiency parameters of capital. The calibration routine changed over the years. I am not sure how it works actually. But there is some capital stock specific information included: either a capital price that can be determined based on an assumed capital income share and a capital coefficient (K/Y) or a given capital stock trajectory that is based on given GDP and K/Y.
How are the 2005 non-ESM capital stocks determined in REMIND? are they an input data to which the variable is fixed?
Non-ESM capital (everything in vm_cesIO(ppfKap)
is an input into the calibration from file f29_capitalQuantity.cs4r
. The variable is not fixed, but as it is a calibration target, it will pretty much hit the prescribed values by the time the calibration is done.
Calibration input:
$ diff -y <( dumpgdx SSP2EU-AMT-calibrate_2022-11-26_00.07.54/fulldata.gdx f29_capitalQuantity '20[01].,EUR,gdp_SSP2EU,kap[^a-z]' | grep -v '^2000' ) <( dumpgdx SSP2EU-AMT-calibrate_2022-12-03_00.10.49/fulldata.gdx f29_capitalQuantity '20[01].,EUR,gdp_SSP2EU,kap[^a-z]' | grep -v '^2000' )
2005 EUR gdp_SSP2EU kap 45.675646 | 2005 EUR gdp_SSP2EU kap 63.485514
2005 EUR gdp_SSP2EU kap_cement 0.003931 | 2005 EUR gdp_SSP2EU kap_cement 0.007092
2005 EUR gdp_SSP2EU kap_chemicals 0.009574 | 2005 EUR gdp_SSP2EU kap_chemicals 0.019769
2005 EUR gdp_SSP2EU kap_steel_primary 0.002434 | 2005 EUR gdp_SSP2EU kap_steel_primary 0.005806
2005 EUR gdp_SSP2EU kap_steel_secondary 0.00059 | 2005 EUR gdp_SSP2EU kap_steel_secondary 0.001134
2005 EUR gdp_SSP2EU kap_otherInd 2.220898 | 2005 EUR gdp_SSP2EU kap_otherInd 0.119269
2010 EUR gdp_SSP2EU kap 48.12552 | 2010 EUR gdp_SSP2EU kap 66.602287
2010 EUR gdp_SSP2EU kap_cement 0.004168 | 2010 EUR gdp_SSP2EU kap_cement 0.00659
2010 EUR gdp_SSP2EU kap_chemicals 0.011151 | 2010 EUR gdp_SSP2EU kap_chemicals 0.020252
2010 EUR gdp_SSP2EU kap_steel_primary 0.002324 | 2010 EUR gdp_SSP2EU kap_steel_primary 0.00584
2010 EUR gdp_SSP2EU kap_steel_secondary 0.000567 | 2010 EUR gdp_SSP2EU kap_steel_secondary 0.001129
2010 EUR gdp_SSP2EU kap_otherInd 2.619937 | 2010 EUR gdp_SSP2EU kap_otherInd 0.123344
2015 EUR gdp_SSP2EU kap 50.775005 | 2015 EUR gdp_SSP2EU kap 69.961441
2015 EUR gdp_SSP2EU kap_cement 0.004435 | 2015 EUR gdp_SSP2EU kap_cement 0.006569
2015 EUR gdp_SSP2EU kap_chemicals 0.013394 | 2015 EUR gdp_SSP2EU kap_chemicals 0.020857
2015 EUR gdp_SSP2EU kap_steel_primary 0.002143 | 2015 EUR gdp_SSP2EU kap_steel_primary 0.005829
2015 EUR gdp_SSP2EU kap_steel_secondary 0.000532 | 2015 EUR gdp_SSP2EU kap_steel_secondary 0.001088
2015 EUR gdp_SSP2EU kap_otherInd 3.183333 | 2015 EUR gdp_SSP2EU kap_otherInd 0.12639
Calibration output:
$ diff -y <( dumpgdx SSP2EU-AMT-calibrate_2022-11-26_00.07.54/fulldata.gdx vm_cesIO '20[01].,EUR,kap.*,L\>' ) <( dumpgdx SSP2EU-AMT-calibrate_2022-12-03_00.10.49/fulldata.gdx vm_cesIO '20[01].,EUR,kap.*,L\>' )
2005 EUR kap L 45.9529912876245 | 2005 EUR kap L 66.9989542470875
2005 EUR kap_cement L 0.00415857769748 | 2005 EUR kap_cement L 0.00750257772336
2005 EUR kap_chemicals L 0.01012826834792 | 2005 EUR kap_chemicals L 0.02091348829852
2005 EUR kap_steel_primary L 0.00257491175672 | 2005 EUR kap_steel_primary L 0.00614212722248
2005 EUR kap_steel_secondary L 0.0006241569172 | 2005 EUR kap_steel_secondary L 0.00119965075272
2005 EUR kap_otherInd L 2.34947262558584 | 2005 EUR kap_otherInd L 0.12617384975852
2010 EUR kap L 48.5129788171281 | 2010 EUR kap L 69.3227091058502
2010 EUR kap_cement L 0.00428123645226762 | 2010 EUR kap_cement L 0.00705534999191206
2010 EUR kap_chemicals L 0.0115592347538713 | 2010 EUR kap_chemicals L 0.0208112019814236
2010 EUR kap_steel_primary L 0.00244786506330071 | 2010 EUR kap_steel_primary L 0.00610218654641758
2010 EUR kap_steel_secondary L 0.000542297248669287 | 2010 EUR kap_steel_secondary L 0.001116553251286
2010 EUR kap_otherInd L 2.71558624187911 | 2010 EUR kap_otherInd L 0.125625986496384
2015 EUR kap L 50.3772977626294 | 2015 EUR kap L 74.3373490709143
2015 EUR kap_cement L 0.00476266340122844 | 2015 EUR kap_cement L 0.00723074412570194
2015 EUR kap_chemicals L 0.0146241479955457 | 2015 EUR kap_chemicals L 0.0224945594237425
2015 EUR kap_steel_primary L 0.00238542485567444 | 2015 EUR kap_steel_primary L 0.00643101382013324
2015 EUR kap_steel_secondary L 0.000595162919914336 | 2015 EUR kap_steel_secondary L 0.00120207297749892
2015 EUR kap_otherInd L 3.52363935964865 | 2015 EUR kap_otherInd L 0.135921445082906
So that changed between
$ ag mrremind rev6.316_62eff8f7_remind/f29_capitalQuantity.cs4r rev6.324_62eff8f7_remind/f29_capitalQuantity.cs4r
rev6.316_62eff8f7_remind/f29_capitalQuantity.cs4r
3:* origin: calcOutput(type = "Capital", file = "f29_capitalQuantity.cs4r", round = 6) (madrat 2.19.3 | mrremind 0.141.1)
rev6.324_62eff8f7_remind/f29_capitalQuantity.cs4r
3:* origin: calcOutput(type = "Capital", file = "f29_capitalQuantity.cs4r", round = 6) (madrat 2.20.6 | mrremind 0.147.1)
pik-piam/mrremind#307 and pik-piam/mrremind#328.
That makes for 18 merge requests into mrremind
that could be responsible:
$ echo -e 'hash|merge request|version|message\n---|---|---|---'; for h in $( git log --format='%h' --abbrev-commit 1242136^1..298465a --grep "Merge pull request" ); do v=$( git show "$h":./DESCRIPTION | sed -n 's/Version: //p' ); git --no-pager log -1 --format='%h;%s;%b' "$h" | sed "s/;Merge pull request #\([0-9]*\).*;/ | [\1](https:\/\/github.com\/pik-piam\/mrremind\/pull\/\1) | $v |/"; done
hash | merge request | version | message |
---|---|---|---|
298465a | 328 | 0.147.1 | Update EEA Projections |
dd270f5 | 327 | 0.147.0 | add new data source: Ember |
a022059 | 321 | 0.146.0 | cement production matches historic data |
da0a7cc | 326 | 0.145.4 | adjust weights in 'calcEmissionFactorsFeedstocks' to avoid zeros in a… |
57fc7ad | 323 | 0.145.4 | bug fix |
2d20da2 | 320 | 0.145.3 | Avoid duplication of scenarios due to imprecise filtering. |
8a69d64 | 322 | 0.145.3 | fix error in readTCdamageKrichene |
7cbdd29 | 319 | 0.145.3 | Corrections for tropical cyclone input data now not dependent on SSP … |
e0a8968 | 318 | 0.145.2 | Include SE Electricity variables from AGEB in historical.mif |
fe97a4f | 317 | 0.145.1 | clean various roxygen warnings |
d7bab45 | 306 | 0.145.0 | integrate industry EEK calculation, update cement production source |
9b026c1 | 316 | 0.144.0 | SSP2_lowEn for all standard scenario combinations |
176c14b | 315 | 0.143.1 | bump UNFCCC to 2022 data |
e48a8ef | 313 | 0.143.0 | Calculate emission factors for feedstocks in the chemicals industry |
d06a453 | 311 | 0.142.0 | add calcCement(), combining data from readvanRuijven2016() and readUSGS(cement) |
847adc3 | 310 | 0.141.4 | Correct mapping |
073c0f3 | 309 | 0.141.3 | Allow more scenario combinations |
afc4939 | 308 | 0.141.2 | Bugfix calc scenario combinations EDGE-Transport |
1242136 | 307 | 0.141.1 | I allow myself to merge this directly as it just fixes a typo. |
I guess I will look at pik-piam/mrremind#306 …
Thanks Michaja. Thus, it is quite likely that the jumps are caused by a change of capital stock input data with non-ESM capital showing the largest difference. Could anybody already detect in which of the considered PRs this change is hidden?
Aaand we're fucked.
I think we moved past pik-piam/mrremind#314, the switch to the new IEA data, which is kind of a point-of-no-return for testing older mrremind
revisions. @fbenke-pik, can you confirm? If yes, what possibilities exist to run the 2016 IEA data?
We unfortunately don't have a switch to hop between old and new data, so I'd agree with the point-of-no-return phrasing. We could implement it if needed, but would involve extensive validation of the interplay of old data with all the libraries (as it affects many calc functions).
Other than that, I can only think of jumping back to the latest mrremind/mrcommons versions before the switch.
Thus, it is quite likely that the jumps are caused by a change of capital stock input data with non-ESM capital showing the largest difference. Could anybody already detect in which of the considered PRs this change is hidden?
Well, pik-piam/mrremind#306 seems to be the only thing touching on the capital stocks, judging from the merge request messages (which are not always … comprehensive, shall we say?)
The energy efficiency capital (EEK) is subtracted from the general capital stock, to avoid double counting. There was a bug with too much EEK, but the jump in kap
is way more than the decrease in EEK.
I'm not sure if something, and if so, what, in pik-piam/mrremind#306 could have caused the difference in kap
.
But as it is questionable whether or when we will get a definitive answer on what caused the change, we should maybe think along different lines, too.
So, why did 2005 investments not change? If we are not married to the precise consumption path, only to the idea of an kink-free consumption path, having "smooth" investments from 2005 on might be the way to go.
I agree with Michaja's last proposal (smoothing 2005 investments). Investments in 2005 are currently fixed to an "empirical" value (which we may have to check). It probably does not fit to the new initial capital stock. HOWEVER, where does the change in the capital stock estimation comes from ???
I agree with Michaja's last proposal (smoothing 2005 investments). Investments in 2005 are currently fixed to an "empirical" value (which we may have to check). It probably does not fit to the new initial capital stock.
Well, it's "real" world data (PWT investment shares). And it didn't really fit the the investments REMIND is doing before hand, but it did work out to a smooth behaviour.
HOWEVER, where does the change in the capital stock estimation comes from ???
Guess I will dissect that.
Guess I will dissect that.
Before anything industry-related is happening, cap_macro[EUR,2005,'gdp_SSP2EU.kap']
is at 63.2. So it's not the EEK …
Is there anything I can do at the moment? Staring at the source code doesn't seem to work, and I miss the "how to debug input data errors" tutorial that would help me where to start :)
Michaja, thank you very much for digging deep and localizing the source of the problem. Now Macro has to do its homework.
FYI @robertpietzcker, @mleimbach, @LaviniaBaumstark, @aodenweller, @fschreyer.
End of November/beginning of December 2022, the calibration changed substantially, affecting consumption, investment and capital stock behavior.
Capital stocks increased:
Consumption shows a significant drop after 2005:![image](https://user-images.githubusercontent.com/90761609/229121027-1280eb54-cc6f-4779-a4eb-05788059a054.png)
Investment is higher:![image](https://user-images.githubusercontent.com/90761609/229121121-087bd9cf-931e-41a4-baf8-d2c880c7fe21.png)
Somehow, the CES parameters changed substantially, for example in REF, but also in EUR, as extracted with the small script attached. The relevant AMT runs in
/p/projects/remind/modeltests/output/
are:SSP2EU-AMT-Base_2022-11-26_00.08.52
vs.SSP2EU-AMT-Base_2022-12-03_00.12.29
SSP2EU-AMT-calibrate_2022-11-26_00.07.54
vs.SSP2EU-AMT-calibrate_2022-12-03_00.10.49
Dear colleagues, it would be awesome if someone with a better intuition about what affects the calibration would have a look at this PR that I suspect may be causing the issue of consumption drop and investment spike in the 2010–2020 years, see attached document.
Somehow, the CES parameter changed substantially, for example in REF, but also in EUR, as extracted with the small script attached. The relevant runs are
/p/projects/remind/modeltests/output/SSP2EU-AMT-Base_2022-11-26_00.08.52
andSSP2EU-AMT-Base_2022-12-03_00.12.29
, respectively theSSP2EU-AMT-calibrate_2022-11-26_00.07.54
/SSP2EU-AMT-calibrate_2022-12-03_00.10.49
runs.A full compareScenarios2 is available at
/p/tmp/oliverr/remind/compScen-consumption_issue-2023-03-31_13.11.07-H12.pdf
.Here is a comparison of all the changes between the
SSP2EU-AMT-Base_2022-11-26
run and the2022-12-03
run: https://github.com/remindmodel/remind/compare/81264e35..a3eb6ac7?w=1My suspicion is this PR, but I'm not entirely sure.