search

pik-piam / magpie4

R package | MAgPIE outputs R package for MAgPIE version 4.x
GNU Lesser General Public License v3.0
2 stars 24 forks source link

Failing summation checks #62

Open orichters opened 7 months ago

orichters commented 7 months ago

Dear MAgPIE colleagues, while debugging the AR6 reporting for REMIND-MAgPIE, I also checked whether the MAgPIE reporting is internally consistent according to the + notation. Turns out it isn't. The Resources|Land Cover issues are probably because irrigated and rainfed should be on a different summation using ++. For the rest, I don't know. Maybe worth having a look.

Demand|+|Bioenergy 1 >
   + Demand|Bioenergy|++|1st generation
   + Demand|Bioenergy|++|2nd generation
   + Demand|Bioenergy|++|Overproduction
   + Demand|Bioenergy|++|Traditional Burning
Relative difference between 97.3% and 98.4%, absolute difference up to 3869 Mt DM/yr.

Demand|+|Processing 1 <
   + Demand|Processing|++|Breeding
   + Demand|Processing|++|Distiilling
   + Demand|Processing|++|Extracting
   + Demand|Processing|++|Fermentation
   + Demand|Processing|++|Ginning
   + Demand|Processing|++|Milling
   + Demand|Processing|++|Refining
   + Demand|Processing|++|Substitutes for Byproducts
Relative difference between -505% and -11.2%, absolute difference up to 1247 Mt DM/yr.

Emissions before technical mitigation|CH4|Land|+|Agriculture >
   + Emissions before technical mitigation|CH4|Land|Agriculture|+|Animal waste management
   + Emissions before technical mitigation|CH4|Land|Agriculture|+|Enteric fermentation
   + Emissions before technical mitigation|CH4|Land|Agriculture|+|Rice
Relative difference between 1.19% and 60.2%, absolute difference up to 25.4 Mt CH4/yr.

Emissions before technical mitigation|N2O|Land|+|Agriculture >
   + Emissions before technical mitigation|N2O|Land|Agriculture|+|Agricultural Soils
   + Emissions before technical mitigation|N2O|Land|Agriculture|+|Animal Waste Management
Relative difference between 1.19% and 43.2%, absolute difference up to 0.598 Mt N2O/yr.

Emissions|CO2|Land|+|Land-use Change 1 >
   + Emissions|CO2|Land|Land-use Change|++|Above Ground Carbon
   + Emissions|CO2|Land|Land-use Change|++|Below Ground Carbon
Relative difference between -7917% and 1926%, absolute difference up to 1937 Mt CO2/yr.

Emissions|CO2|Land|Cumulative|Land-use Change|+|Gross LUC <>
   + Emissions|CO2|Land|Cumulative|Land-use Change|Gross LUC|+|Deforestation
   + Emissions|CO2|Land|Cumulative|Land-use Change|Gross LUC|+|Forest Degradation
   + Emissions|CO2|Land|Cumulative|Land-use Change|Gross LUC|+|Other land conversion
Relative difference between -4447% and 6627%, absolute difference up to 148 Gt CO2.

Emissions|CO2|Land|Land-use Change|+|Gross LUC <>
   + Emissions|CO2|Land|Land-use Change|Gross LUC|+|Deforestation
   + Emissions|CO2|Land|Land-use Change|Gross LUC|+|Forest Degradation
   + Emissions|CO2|Land|Land-use Change|Gross LUC|+|Other land conversion
Relative difference between -1322212% and 257721%, absolute difference up to 1732 Mt CO2/yr.

Processing|Raw material|Secondary products|+|Brans <
   + Processing|Raw material|Secondary products|Brans|+|Brans
   + Processing|Raw material|Secondary products|Brans|+|Cotton seed
   + Processing|Raw material|Secondary products|Brans|+|Forage
   + Processing|Raw material|Secondary products|Brans|+|Fruits Vegetables Nuts
   + Processing|Raw material|Secondary products|Brans|+|Groundnuts
   + Processing|Raw material|Secondary products|Brans|+|Maize
   + Processing|Raw material|Secondary products|Brans|+|Molasses
   + Processing|Raw material|Secondary products|Brans|+|Oilcakes
   + Processing|Raw material|Secondary products|Brans|+|Oilpalms
   + Processing|Raw material|Secondary products|Brans|+|Oils
   + Processing|Raw material|Secondary products|Brans|+|Other oil crops incl rapeseed
   + Processing|Raw material|Secondary products|Brans|+|Potatoes
   + Processing|Raw material|Secondary products|Brans|+|Rice
   + Processing|Raw material|Secondary products|Brans|+|Short rotation grasses
   + Processing|Raw material|Secondary products|Brans|+|Short rotation trees
   + Processing|Raw material|Secondary products|Brans|+|Soybean
   + Processing|Raw material|Secondary products|Brans|+|Sugar
   + Processing|Raw material|Secondary products|Brans|+|Sugar beet
   + Processing|Raw material|Secondary products|Brans|+|Sugar cane
   + Processing|Raw material|Secondary products|Brans|+|Sunflower
   + Processing|Raw material|Secondary products|Brans|+|Temperate cereals
   + Processing|Raw material|Secondary products|Brans|+|Tropical cereals
   + Processing|Raw material|Secondary products|Brans|+|Tropical roots
Relative difference between -Inf% and -2771%, absolute difference up to 252 Mt DM/yr.

Processing|Raw material|Secondary products|+|Molasses <
   + Processing|Raw material|Secondary products|Molasses|+|Brans
   + Processing|Raw material|Secondary products|Molasses|+|Cotton seed
   + Processing|Raw material|Secondary products|Molasses|+|Forage
   + Processing|Raw material|Secondary products|Molasses|+|Fruits Vegetables Nuts
   + Processing|Raw material|Secondary products|Molasses|+|Groundnuts
   + Processing|Raw material|Secondary products|Molasses|+|Maize
   + Processing|Raw material|Secondary products|Molasses|+|Molasses
   + Processing|Raw material|Secondary products|Molasses|+|Oilcakes
   + Processing|Raw material|Secondary products|Molasses|+|Oilpalms
   + Processing|Raw material|Secondary products|Molasses|+|Oils
   + Processing|Raw material|Secondary products|Molasses|+|Other oil crops incl rapeseed
   + Processing|Raw material|Secondary products|Molasses|+|Potatoes
   + Processing|Raw material|Secondary products|Molasses|+|Rice
   + Processing|Raw material|Secondary products|Molasses|+|Short rotation grasses
   + Processing|Raw material|Secondary products|Molasses|+|Short rotation trees
   + Processing|Raw material|Secondary products|Molasses|+|Soybean
   + Processing|Raw material|Secondary products|Molasses|+|Sugar
   + Processing|Raw material|Secondary products|Molasses|+|Sugar beet
   + Processing|Raw material|Secondary products|Molasses|+|Sugar cane
   + Processing|Raw material|Secondary products|Molasses|+|Sunflower
   + Processing|Raw material|Secondary products|Molasses|+|Temperate cereals
   + Processing|Raw material|Secondary products|Molasses|+|Tropical cereals
   + Processing|Raw material|Secondary products|Molasses|+|Tropical roots
Relative difference between -70137% and -296%, absolute difference up to 121 Mt DM/yr.

Processing|Raw material|Secondary products|+|Oilcakes <
   + Processing|Raw material|Secondary products|Oilcakes|+|Brans
   + Processing|Raw material|Secondary products|Oilcakes|+|Cotton seed
   + Processing|Raw material|Secondary products|Oilcakes|+|Forage
   + Processing|Raw material|Secondary products|Oilcakes|+|Fruits Vegetables Nuts
   + Processing|Raw material|Secondary products|Oilcakes|+|Groundnuts
   + Processing|Raw material|Secondary products|Oilcakes|+|Maize
   + Processing|Raw material|Secondary products|Oilcakes|+|Molasses
   + Processing|Raw material|Secondary products|Oilcakes|+|Oilcakes
   + Processing|Raw material|Secondary products|Oilcakes|+|Oilpalms
   + Processing|Raw material|Secondary products|Oilcakes|+|Oils
   + Processing|Raw material|Secondary products|Oilcakes|+|Other oil crops incl rapeseed
   + Processing|Raw material|Secondary products|Oilcakes|+|Potatoes
   + Processing|Raw material|Secondary products|Oilcakes|+|Rice
   + Processing|Raw material|Secondary products|Oilcakes|+|Short rotation grasses
   + Processing|Raw material|Secondary products|Oilcakes|+|Short rotation trees
   + Processing|Raw material|Secondary products|Oilcakes|+|Soybean
   + Processing|Raw material|Secondary products|Oilcakes|+|Sugar
   + Processing|Raw material|Secondary products|Oilcakes|+|Sugar beet
   + Processing|Raw material|Secondary products|Oilcakes|+|Sugar cane
   + Processing|Raw material|Secondary products|Oilcakes|+|Sunflower
   + Processing|Raw material|Secondary products|Oilcakes|+|Temperate cereals
   + Processing|Raw material|Secondary products|Oilcakes|+|Tropical cereals
   + Processing|Raw material|Secondary products|Oilcakes|+|Tropical roots
Relative difference between -Inf% and -Inf%, absolute difference up to 366 Mt DM/yr.

Processing|Raw material|Secondary products|+|Oils <
   + Processing|Raw material|Secondary products|Oils|+|Brans
   + Processing|Raw material|Secondary products|Oils|+|Cotton seed
   + Processing|Raw material|Secondary products|Oils|+|Forage
   + Processing|Raw material|Secondary products|Oils|+|Fruits Vegetables Nuts
   + Processing|Raw material|Secondary products|Oils|+|Groundnuts
   + Processing|Raw material|Secondary products|Oils|+|Maize
   + Processing|Raw material|Secondary products|Oils|+|Molasses
   + Processing|Raw material|Secondary products|Oils|+|Oilcakes
   + Processing|Raw material|Secondary products|Oils|+|Oilpalms
   + Processing|Raw material|Secondary products|Oils|+|Oils
   + Processing|Raw material|Secondary products|Oils|+|Other oil crops incl rapeseed
   + Processing|Raw material|Secondary products|Oils|+|Potatoes
   + Processing|Raw material|Secondary products|Oils|+|Rice
   + Processing|Raw material|Secondary products|Oils|+|Short rotation grasses
   + Processing|Raw material|Secondary products|Oils|+|Short rotation trees
   + Processing|Raw material|Secondary products|Oils|+|Soybean
   + Processing|Raw material|Secondary products|Oils|+|Sugar
   + Processing|Raw material|Secondary products|Oils|+|Sugar beet
   + Processing|Raw material|Secondary products|Oils|+|Sugar cane
   + Processing|Raw material|Secondary products|Oils|+|Sunflower
   + Processing|Raw material|Secondary products|Oils|+|Temperate cereals
   + Processing|Raw material|Secondary products|Oils|+|Tropical cereals
   + Processing|Raw material|Secondary products|Oils|+|Tropical roots
Relative difference between -Inf% and -Inf%, absolute difference up to 348 Mt DM/yr.

Processing|Raw material|Secondary products|+|Sugar <
   + Processing|Raw material|Secondary products|Sugar|+|Brans
   + Processing|Raw material|Secondary products|Sugar|+|Cotton seed
   + Processing|Raw material|Secondary products|Sugar|+|Forage
   + Processing|Raw material|Secondary products|Sugar|+|Fruits Vegetables Nuts
   + Processing|Raw material|Secondary products|Sugar|+|Groundnuts
   + Processing|Raw material|Secondary products|Sugar|+|Maize
   + Processing|Raw material|Secondary products|Sugar|+|Molasses
   + Processing|Raw material|Secondary products|Sugar|+|Oilcakes
   + Processing|Raw material|Secondary products|Sugar|+|Oilpalms
   + Processing|Raw material|Secondary products|Sugar|+|Oils
   + Processing|Raw material|Secondary products|Sugar|+|Other oil crops incl rapeseed
   + Processing|Raw material|Secondary products|Sugar|+|Potatoes
   + Processing|Raw material|Secondary products|Sugar|+|Rice
   + Processing|Raw material|Secondary products|Sugar|+|Short rotation grasses
   + Processing|Raw material|Secondary products|Sugar|+|Short rotation trees
   + Processing|Raw material|Secondary products|Sugar|+|Soybean
   + Processing|Raw material|Secondary products|Sugar|+|Sugar
   + Processing|Raw material|Secondary products|Sugar|+|Sugar beet
   + Processing|Raw material|Secondary products|Sugar|+|Sugar cane
   + Processing|Raw material|Secondary products|Sugar|+|Sunflower
   + Processing|Raw material|Secondary products|Sugar|+|Temperate cereals
   + Processing|Raw material|Secondary products|Sugar|+|Tropical cereals
   + Processing|Raw material|Secondary products|Sugar|+|Tropical roots
Relative difference between -Inf% and -1630%, absolute difference up to 706 Mt DM/yr.

Productivity|Yield|+|Bioenergy crops <
   + Productivity|Yield|Bioenergy crops|+|Short rotation grasses
   + Productivity|Yield|Bioenergy crops|+|Short rotation trees
Relative difference between -194% and -16.8%, absolute difference up to 52.2 t DM/ha.

Productivity|Yield|+|Crops <
   + Productivity|Yield|Crops|+|Cereals
   + Productivity|Yield|Crops|+|Oil crops
   + Productivity|Yield|Crops|+|Other crops
   + Productivity|Yield|Crops|+|Sugar crops
Relative difference between -2840% and -168%, absolute difference up to 94.2 t DM/ha.

Productivity|Yield|Crops|+|Cereals <
   + Productivity|Yield|Crops|Cereals|+|Maize
   + Productivity|Yield|Crops|Cereals|+|Rice
   + Productivity|Yield|Crops|Cereals|+|Temperate cereals
   + Productivity|Yield|Crops|Cereals|+|Tropical cereals
Relative difference between -1179% and -45.1%, absolute difference up to 38.6 t DM/ha.

Productivity|Yield|Crops|+|Oil crops <
   + Productivity|Yield|Crops|Oil crops|+|Cotton seed
   + Productivity|Yield|Crops|Oil crops|+|Groundnuts
   + Productivity|Yield|Crops|Oil crops|+|Oilpalms
   + Productivity|Yield|Crops|Oil crops|+|Other oil crops incl rapeseed
   + Productivity|Yield|Crops|Oil crops|+|Soybean
   + Productivity|Yield|Crops|Oil crops|+|Sunflower
Relative difference between -1096% and -49.1%, absolute difference up to 31.4 t DM/ha.

Productivity|Yield|Crops|+|Other crops <
   + Productivity|Yield|Crops|Other crops|+|Fruits Vegetables Nuts
   + Productivity|Yield|Crops|Other crops|+|Potatoes
   + Productivity|Yield|Crops|Other crops|+|Pulses
   + Productivity|Yield|Crops|Other crops|+|Tropical roots
Relative difference between -1386% and -143%, absolute difference up to 48.4 t DM/ha.

Productivity|Yield|Crops|+|Sugar crops <
   + Productivity|Yield|Crops|Sugar crops|+|Sugar beet
   + Productivity|Yield|Crops|Sugar crops|+|Sugar cane
Relative difference between -427% and -3.99%, absolute difference up to 73.5 t DM/ha.

Resources|Land Cover|+|Cropland <
   + Resources|Land Cover|Cropland|+|Bioenergy crops
   + Resources|Land Cover|Cropland|+|Crops
   + Resources|Land Cover|Cropland|+|Fallow Cropland
   + Resources|Land Cover|Cropland|+|Forage
   + Resources|Land Cover|Cropland|+|Irrigated
   + Resources|Land Cover|Cropland|+|Rainfed
Relative difference between -100% and -100%, absolute difference up to 1931 million ha.

Resources|Land Cover|Cropland|+|Bioenergy crops <
   + Resources|Land Cover|Cropland|Bioenergy crops|+|irrigated
   + Resources|Land Cover|Cropland|Bioenergy crops|+|rainfed
   + Resources|Land Cover|Cropland|Bioenergy crops|+|Short rotation grasses
   + Resources|Land Cover|Cropland|Bioenergy crops|+|Short rotation trees
Relative difference between -100% and -98.4%, absolute difference up to 92.2 million ha.

Resources|Land Cover|Cropland|+|Crops <
   + Resources|Land Cover|Cropland|Crops|+|Cereals
   + Resources|Land Cover|Cropland|Crops|+|irrigated
   + Resources|Land Cover|Cropland|Crops|+|Oil crops
   + Resources|Land Cover|Cropland|Crops|+|Other crops
   + Resources|Land Cover|Cropland|Crops|+|rainfed
   + Resources|Land Cover|Cropland|Crops|+|Sugar crops
Relative difference between -100% and -100%, absolute difference up to 1710 million ha.

Resources|Land Cover|Cropland|Crops|+|Cereals <
   + Resources|Land Cover|Cropland|Crops|Cereals|+|irrigated
   + Resources|Land Cover|Cropland|Crops|Cereals|+|Maize
   + Resources|Land Cover|Cropland|Crops|Cereals|+|rainfed
   + Resources|Land Cover|Cropland|Crops|Cereals|+|Rice
   + Resources|Land Cover|Cropland|Crops|Cereals|+|Temperate cereals
   + Resources|Land Cover|Cropland|Crops|Cereals|+|Tropical cereals
Relative difference between -100% and -100%, absolute difference up to 1067 million ha.

Resources|Land Cover|Cropland|Crops|+|Oil crops <
   + Resources|Land Cover|Cropland|Crops|Oil crops|+|Cotton seed
   + Resources|Land Cover|Cropland|Crops|Oil crops|+|Groundnuts
   + Resources|Land Cover|Cropland|Crops|Oil crops|+|irrigated
   + Resources|Land Cover|Cropland|Crops|Oil crops|+|Oilpalms
   + Resources|Land Cover|Cropland|Crops|Oil crops|+|Other oil crops incl rapeseed
   + Resources|Land Cover|Cropland|Crops|Oil crops|+|rainfed
   + Resources|Land Cover|Cropland|Crops|Oil crops|+|Soybean
   + Resources|Land Cover|Cropland|Crops|Oil crops|+|Sunflower
Relative difference between -100% and -99.8%, absolute difference up to 302 million ha.

Resources|Land Cover|Cropland|Crops|+|Other crops <
   + Resources|Land Cover|Cropland|Crops|Other crops|+|Fruits Vegetables Nuts
   + Resources|Land Cover|Cropland|Crops|Other crops|+|irrigated
   + Resources|Land Cover|Cropland|Crops|Other crops|+|Potatoes
   + Resources|Land Cover|Cropland|Crops|Other crops|+|Pulses
   + Resources|Land Cover|Cropland|Crops|Other crops|+|rainfed
   + Resources|Land Cover|Cropland|Crops|Other crops|+|Tropical roots
Relative difference between -100% and -100%, absolute difference up to 329 million ha.

Resources|Land Cover|Cropland|Crops|+|Sugar crops <
   + Resources|Land Cover|Cropland|Crops|Sugar crops|+|irrigated
   + Resources|Land Cover|Cropland|Crops|Sugar crops|+|rainfed
   + Resources|Land Cover|Cropland|Crops|Sugar crops|+|Sugar beet
   + Resources|Land Cover|Cropland|Crops|Sugar crops|+|Sugar cane
Relative difference between -100% and -99.8%, absolute difference up to 38.1 million ha.

Code

library(tidyverse); library(quitte); library(piamInterfaces); options(width = 180)
f <- "/p/projects/landuse/tests/magpie/output/weeklyTests_SSP2-NDC/report.mif"
failvar <- checkSummations(f, outputDirectory = NULL, summationsFile = "extractVariableGroups")
FelicitasBeier commented 7 months ago

The "Resources|Land Cover" and "Yields" ones are solved now.

For those of you who are working on the other ones (@mscrawford and @caviddhen and @deleip and @k4rst3ns), if you are checking whether your fix worked, you'll need to run piamInterfaces::summationChecks("report.mif"), but don't select AR6 (number 2) because there everything actually works. Select 1: extractVariableGroups, then the remaining warnings appear.