Open 0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q opened 2 years ago
fschreyer
commented
2 years ago Ok, I think this is an issue which comes from the EDGE-T reporting and not from the standard reporting which is run by convGDX2MIF. The only variable existing in the standard reporting is FE|Transport|Freight|+|Liquids which will become FE|Transport|Freight in the mif without plus. So, I think @Loisel would need to take over.
Loisel
commented
2 years ago This is present at least since 2021-10-09, but after 2021-10-02
Are you sure about this @0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q ? When looking at the libraries, the duplicated variables should be there for 8 months already. Maybe some mechanism previously selected one of them for the output.
Loisel
commented
2 years ago Here is the line where @MariannaR most likely introduced the duplicates https://github.com/pik-piam/remind2/blame/master/R/reportEDGETransport.R#L87
0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q
commented
2 years ago This is present at least since 2021-10-09, but after 2021-10-02
Are you sure about this @0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q ? When looking at the libraries, the duplicated variables should be there for 8 months already. Maybe some mechanism previously selected one of them for the output.
I'm sure about SSP2-AMT-Base_2021-10-09_00.00.24 having duplicated variables in the reporting and SSP2-AMT-Base_2021-10-02_00.00.21 not having it.
And I'm reasonably sure that only remind2 had relevant changes in the meantime:
$ diff -y <( sed -n '/^==== installed package versions =====/,/^=====/ { /^ \?[0-9]\+/ p }' /p/projects/remind/modeltests/output/SSP2-AMT-Base_2021-10-02_00.00.21/log.txt ) <( sed -n '/^==== installed package versions =====/,/^=====/ { /^ \?[0-9]\+/ p }' /p/projects/remind/modeltests/output/SSP2-AMT-Base_2021-10-09_00.00.24/log.txt )
1 data.table 1.14.0 1 data.table 1.14.0
2 devtools 2.4.2 2 devtools 2.4.2
3 dplyr 1.0.7 3 dplyr 1.0.7
4 edgeTrpLib 0.1.48 4 edgeTrpLib 0.1.48
5 flexdashboard 0.5.2 5 flexdashboard 0.5.2
6 gdx 1.51.4 6 gdx 1.51.4
7 gdxdt 0.1.0 7 gdxdt 0.1.0
8 gdxrrw 1.0.2 8 gdxrrw 1.0.2
9 ggplot2 3.3.5 9 ggplot2 3.3.5
10 gtools 3.9.2 10 gtools 3.9.2
11 lucode 3.21.0 11 lucode 3.21.0
12 luplot 3.51.2 12 luplot 3.51.2
13 luscale 2.23.2 13 luscale 2.23.2
14 magclass 6.0.9 14 magclass 6.0.9
15 magpie 0.2266.1 15 magpie 0.2266.1
16 mip 0.131.4 | 16 mip 0.132.3
17 mrremind 0.80.0 | 17 mrremind 0.82.0
18 mrvalidation 2.30.3 | 18 mrvalidation 2.30.4
19 optparse 1.6.6 19 optparse 1.6.6
20 plotly 4.9.4.1 20 plotly 4.9.4.1
21 remind 36.185.2 21 remind 36.185.2
22 remind2 1.46.0 | 22 remind2 1.46.1
23 rlang 0.4.11 23 rlang 0.4.11
24 rmndt 0.5.4 24 rmndt 0.5.4
25 tidyverse 1.3.1 25 tidyverse 1.3.1
26 data.table 1.14.0 26 data.table 1.14.0
27 dplyr 1.0.6 27 dplyr 1.0.6
28 ggplot2 3.3.3 28 ggplot2 3.3.3
29 gtools 3.8.2 29 gtools 3.8.2
30 methods 3.6.3 30 methods 3.6.3
31 parallel 3.6.3 31 parallel 3.6.3
32 rlang 0.4.11 32 rlang 0.4.11
33 tidyverse 1.3.1 33 tidyverse 1.3.1
34 tools 3.6.3 34 tools 3.6.3 And I'm sure again that between those two runs, there was only one request merged into remind2:
$ git log --format="%C(yellow)%h %C(cyan)%an %Creset%s" 49abbfdf^..578d2196
578d219 Felix Schreyer update library version
7e23c93 Felix Schreyer add EU27 region to edge transport reporting
3fe68ed Felix Schreyer bugfix in steel production reporting
2a31aca fschreyer Merge pull request #115 from fschreyer/industry_CCS
89d34bc Felix Schreyer correct some code descriptions
49abbfd Felix Schreyer update library versionSo maybe bisecting 2a31aca to 578d219 will shed light on this.
Variables
are reported twice in reporting, with different values, breaking further analysis. This is present at least since 2021-10-09, but after 2021-10-02
which points towards #116.