search

SLU-foodsystems / safad

A web-based tool to compute the environmental impacts of a diet, led by Elin Röös.
https://safad.se
MIT License
1 stars 0 forks source link

Sums of shares in RPC Origin and Waste files do not always add up to 100% #13

Closed wanecek closed 5 days ago

wanecek commented 1 month ago

We recently found that the sum of shares in the footprints files do not necessarily add up to 1 (=100%). Initially, we thought this was only in the file of Poland, but it seems to be the case for all/most files.

wanecek commented 1 month ago

After removing exact duplicate rows, we have the following rpc-codes with invalid sums (1+-0.025).

We can see that several of these have the same SUA code. Since we calculated baseline of the shares from Kastner et al. -> sua -> rpc, some of these are repetitions of the same mistake (e.g. Other fruits, n.e.c.).

DE.csv

RPC Code,RPC Name,SUA code,SUA name,Sum of shares in file
A.02.02.001,Garlic,01252,green garlic,0.726
A.02.03.005,Okra,01359.90,"Other fruits, n.e.c.",0.795
A.02.05.014,Grape leaves,01359.90,"Other fruits, n.e.c.",0.795
A.02.05.017,Oraches leaves,01359.90,"Other fruits, n.e.c.",0.795
A.02.05.022,Alfalfa sprouts,01709.90,other pulses n.e.c.,0.539
A.02.06.001,Beans (with pods) and similar-,01701,"beans, dry",0.539
A.02.07.008,Bamboo shoots,01359.90,"Other fruits, n.e.c.",0.795
A.02.07.009,Palm hearts,01359.90,"Other fruits, n.e.c.",0.795
A.02.10.002,"Tea (dried leaves and stalks, fermented or otherwise of Camellia sinensis), decaffeinated",01620,Tea leaves,0.802
A.02.15.001,Common mushrooms,01270,Mushrooms and truffles,1.588
A.02.15.002,Oyster mushroom (Pleurotus ostreatus),01270,Mushrooms and truffles,0.588
A.02.15.003,Shiitake,01270,Mushrooms and truffles,0.588
A.02.16.002,Honey mushroom,01270,Mushrooms and truffles,1.588
A.02.16.003,Ceps,01270,Mushrooms and truffles,1.588
A.02.16.004,Truffles,01270,Mushrooms and truffles,1.588
A.02.16.005,Morels,01270,Mushrooms and truffles,1.588
A.02.16.006,Chanterelles,01270,Mushrooms and truffles,1.588
A.02.16.007,Saffron milk cap,01270,Mushrooms and truffles,1.588
A.04.01.001,Beans (fresh seeds without pods) and similar-,01701,"Beans, dry",0.539
A.04.02.001,Beans (dry) and similar-,01701,"Beans, dry",0.539
A.04.02.008,Lima beans (dry),01701,"Beans, dry",0.539
A.04.02.009,Mung bean sprouts,01701,"Beans, dry",0.539
A.04.02.014,Carobs,01709.90,Other pulses n.e.c.,0.539
A.05.01.001,Grapefruits,01321,Pomelos and grapefruits,0.747
A.05.01.006,Pomelo (Citrus grandis),01321,Pomelos and grapefruits,0.747
A.05.02.001,Apples,01341,Apples,0.825
A.05.02.004,Medlars,01359.90,"Other fruits, n.e.c.",0.795
A.05.02.005,Loquats,01359.90,"Other fruits, n.e.c.",0.795
A.05.02.007,Sorb fruits,01341,Apples,0.825
A.05.05.003,Palmfruit (Elaeis guineensis),01491.01,oil palm fruit,2
A.05.06.005,Carambolas,01359.90,"Other fruits, n.e.c.",0.795
A.05.06.009,Litchis,01359.90,"Other fruits, n.e.c.",0.795
A.05.06.010,Passionfruits,01359.90,"Other fruits, n.e.c.",0.795
A.05.06.011,Prickly pears,01359.90,"Other fruits, n.e.c.",0.795
A.05.06.018,Granate apples,01359.90,"Other fruits, n.e.c.",0.795
A.05.06.019,Wild sweetsops,01359.90,"Other fruits, n.e.c.",0.795
A.05.06.022,Bread fruit (jackfruit) (Artocarpus altilis),01359.90,"Other fruits, n.e.c.",0.795
A.05.06.030,Chayote (Sechium edule),01359.90,"Other fruits, n.e.c.",0.795
A.05.06.034,Longans,01359.90,"Other fruits, n.e.c.",0.795
A.06.02.005,Pigeon fresh meat,21170.01,"meat of pigeons and other birds n.e.c., fresh, chilled or frozen",0.779
A.10.07,Honey,02910,Honey,0.879

ES

RPC Code,RPC Name,SUA code,SUA name,Sum of shares in file
A.01.02.004,Rye grain,0116,rye,0.747
A.02.05.022,Alfalfa sprouts,01709.90,other pulses n.e.c.,0.836
A.02.06.001,Beans (with pods) and similar-,01701,"beans, dry",0.836
A.02.10.002,"Tea (dried leaves and stalks, fermented or otherwise of Camellia sinensis), decaffeinated",01620,Tea leaves,0.765
A.02.15.001,Common mushrooms,01270,Mushrooms and truffles,1.997
A.02.16.002,Honey mushroom,01270,Mushrooms and truffles,1.997
A.02.16.003,Ceps,01270,Mushrooms and truffles,1.997
A.02.16.004,Truffles,01270,Mushrooms and truffles,1.997
A.02.16.005,Morels,01270,Mushrooms and truffles,1.997
A.02.16.006,Chanterelles,01270,Mushrooms and truffles,1.997
A.02.16.007,Saffron milk cap,01270,Mushrooms and truffles,1.997
A.04.01.001,Beans (fresh seeds without pods) and similar-,01701,"Beans, dry",0.836
A.04.02.001,Beans (dry) and similar-,01701,"Beans, dry",0.836
A.04.02.008,Lima beans (dry),01701,"Beans, dry",0.836
A.04.02.009,Mung bean sprouts,01701,"Beans, dry",0.836
A.04.02.014,Carobs,01709.90,Other pulses n.e.c.,0.836
A.05.05.003,Palmfruit (Elaeis guineensis),01491.01,oil palm fruit,2
A.06.02.005,Pigeon fresh meat,21170.01,"meat of pigeons and other birds n.e.c., fresh, chilled or frozen",0.882
A.10.07,Honey,02910,Honey,0.693

FR

RPC Code,RPC Name,SUA code,SUA name,Sum of shares in file
A.02.03.005,Okra,01359.90,"Other fruits, n.e.c.",0.857
A.02.05.014,Grape leaves,01359.90,"Other fruits, n.e.c.",0.857
A.02.05.017,Oraches leaves,01359.90,"Other fruits, n.e.c.",0.857
A.02.05.022,Alfalfa sprouts,01709.90,other pulses n.e.c.,0.774
A.02.06.001,Beans (with pods) and similar-,01701,"beans, dry",0.774
A.02.07.008,Bamboo shoots,01359.90,"Other fruits, n.e.c.",0.857
A.02.07.009,Palm hearts,01359.90,"Other fruits, n.e.c.",0.857
A.02.10.002,"Tea (dried leaves and stalks, fermented or otherwise of Camellia sinensis), decaffeinated",01620,Tea leaves,0.667
A.02.15.001,Common mushrooms,01270,Mushrooms and truffles,1.878
A.02.15.002,Oyster mushroom (Pleurotus ostreatus),01270,Mushrooms and truffles,0.878
A.02.15.003,Shiitake,01270,Mushrooms and truffles,0.878
A.02.16.002,Honey mushroom,01270,Mushrooms and truffles,1.878
A.02.16.003,Ceps,01270,Mushrooms and truffles,1.878
A.02.16.004,Truffles,01270,Mushrooms and truffles,1.878
A.02.16.005,Morels,01270,Mushrooms and truffles,1.878
A.02.16.006,Chanterelles,01270,Mushrooms and truffles,1.878
A.02.16.007,Saffron milk cap,01270,Mushrooms and truffles,1.878
A.04.01.001,Beans (fresh seeds without pods) and similar-,01701,"Beans, dry",0.774
A.04.02.001,Beans (dry) and similar-,01701,"Beans, dry",0.774
A.04.02.008,Lima beans (dry),01701,"Beans, dry",0.774
A.04.02.009,Mung bean sprouts,01701,"Beans, dry",0.774
A.04.02.014,Carobs,01709.90,Other pulses n.e.c.,0.774
A.04.04.004,Sesame seed (Sesamum indicum syn. S. orientale),01444,Sesame seed,0.737
A.04.04.007,Mustard seed (Brassica nigra),01442,Mustard seed,0.737
A.04.04.009,Pumpkin seeds,01449.90,"Other oil seeds, n.e.c.",0.737
A.04.04.013,Hempseed (Cannabis sativa),01929.02,"True hemp, raw or retted",0.737
A.04.05,Other oilseeds (including elsewhere non-listed seeds),01449.90,"Other oil seeds, n.e.c.",0.737
A.05.01.001,Grapefruits,01321,Pomelos and grapefruits,0.854
A.05.01.006,Pomelo (Citrus grandis),01321,Pomelos and grapefruits,0.854
A.05.02.004,Medlars,01359.90,"Other fruits, n.e.c.",0.857
A.05.02.005,Loquats,01359.90,"Other fruits, n.e.c.",0.857
A.05.05.002,Palm nuts (palmoil kernels),01491.02,palm kernels [only calories],0.737
A.05.05.003,Palmfruit (Elaeis guineensis),01491.01,oil palm fruit,2
A.05.06.005,Carambolas,01359.90,"Other fruits, n.e.c.",0.857
A.05.06.009,Litchis,01359.90,"Other fruits, n.e.c.",0.857
A.05.06.010,Passionfruits,01359.90,"Other fruits, n.e.c.",0.857
A.05.06.011,Prickly pears,01359.90,"Other fruits, n.e.c.",0.857
A.05.06.018,Granate apples,01359.90,"Other fruits, n.e.c.",0.857
A.05.06.019,Wild sweetsops,01359.90,"Other fruits, n.e.c.",0.857
A.05.06.022,Bread fruit (jackfruit) (Artocarpus altilis),01359.90,"Other fruits, n.e.c.",0.857
A.05.06.030,Chayote (Sechium edule),01359.90,"Other fruits, n.e.c.",0.857
A.05.06.034,Longans,01359.90,"Other fruits, n.e.c.",0.857
A.06.02.007,Ratites fresh meat,21170.02,"game meat, fresh, chilled or frozen",0.896
A.06.03.001,Wild boar fresh meat,21170.02,"game meat, fresh, chilled or frozen",0.896
A.06.03.002,"Deer, red fresh meat",21170.02,"game meat, fresh, chilled or frozen",0.896
A.06.03.003,European moose meat,21170.02,"game meat, fresh, chilled or frozen",0.896
A.06.03.004,Reindeer meat,21170.02,"game meat, fresh, chilled or frozen",0.896
A.06.04.001,Pheasant fresh meat,21170.02,"game meat, fresh, chilled or frozen",0.896
A.06.04.003,Mallard fresh meat,21170.02,"game meat, fresh, chilled or frozen",0.896
A.06.04.004,Partridge fresh meat,21170.02,"game meat, fresh, chilled or frozen",0.896
A.06.04.005,Quail fresh meat,21170.02,"game meat, fresh, chilled or frozen",0.896
A.10.07,Honey,02910,Honey,0.752
I.07.01.006.001,Salmons,A0C75,Salmons,2
I.07.01.006.002,Trouts,A029F,Trouts,2
I.07.01.015.001,Catfishes (freshwater),A027J,Catfishes (freshwater),2
I.07.01.015.002,Wolffish,A029Z,Wolffish,2
I.07.01.017.002,seed,A02CG,Whiting,2
I.07.01.017.004,Haddock,A02CA,Haddock,2
I.07.03.001.003,Lumpfish roe,A0F7F,Lumpfish roe,2

GR

RPC Code,RPC Name,SUA code,SUA name,Sum of shares in file
A.02.10.002,"Tea (dried leaves and stalks, fermented or otherwise of Camellia sinensis), decaffeinated",01620,Tea leaves,0.826
A.02.15.001,Common mushrooms,01270,Mushrooms and truffles,1.682
A.02.15.002,Oyster mushroom (Pleurotus ostreatus),01270,Mushrooms and truffles,0.682
A.02.15.003,Shiitake,01270,Mushrooms and truffles,0.682
A.02.16.002,Honey mushroom,01270,Mushrooms and truffles,1.682
A.02.16.003,Ceps,01270,Mushrooms and truffles,1.682
A.02.16.004,Truffles,01270,Mushrooms and truffles,1.682
A.02.16.005,Morels,01270,Mushrooms and truffles,1.682
A.02.16.006,Chanterelles,01270,Mushrooms and truffles,1.682
A.02.16.007,Saffron milk cap,01270,Mushrooms and truffles,1.682
A.05.05.003,Palmfruit (Elaeis guineensis),01491.01,oil palm fruit,2
A.06.02.005,Pigeon fresh meat,21170.01,"meat of pigeons and other birds n.e.c., fresh, chilled or frozen",0.816
A.10.02.001.007,,,,2

HU

RPC Code,RPC Name,SUA code,SUA name,Sum of shares in file
A.02.05.022,Alfalfa sprouts,01709.90,other pulses n.e.c.,0.777
A.02.06.001,Beans (with pods) and similar-,01701,"beans, dry",0.777
A.02.10.002,"Tea (dried leaves and stalks, fermented or otherwise of Camellia sinensis), decaffeinated",01620,Tea leaves,0.818
A.02.15.001,Common mushrooms,01270,Mushrooms and truffles,1.998
A.02.16.002,Honey mushroom,01270,Mushrooms and truffles,1.998
A.02.16.003,Ceps,01270,Mushrooms and truffles,1.998
A.02.16.004,Truffles,01270,Mushrooms and truffles,1.998
A.02.16.005,Morels,01270,Mushrooms and truffles,1.998
A.02.16.006,Chanterelles,01270,Mushrooms and truffles,1.998
A.02.16.007,Saffron milk cap,01270,Mushrooms and truffles,1.998
A.04.01.001,Beans (fresh seeds without pods) and similar-,01701,"Beans, dry",0.777
A.04.02.001,Beans (dry) and similar-,01701,"Beans, dry",0.777
A.04.02.008,Lima beans (dry),01701,"Beans, dry",0.777
A.04.02.009,Mung bean sprouts,01701,"Beans, dry",0.777
A.04.02.014,Carobs,01709.90,Other pulses n.e.c.,0.777
A.05.05.003,Palmfruit (Elaeis guineensis),01491.01,oil palm fruit,2
A.06.01.001,Bovine muscle,21111.02,"meat of cattle boneless, fresh or chilled",0.886
A.06.01.002,Calf fresh meat,21111.02,"meat of cattle boneless, fresh or chilled",0.886
A.06.01.006,Equine muscle,21118.01,"horse meat, fresh or chilled",0.792
A.06.02.005,Pigeon fresh meat,21170.01,"meat of pigeons and other birds n.e.c., fresh, chilled or frozen",0.862

IE

RPC Code,RPC Name,SUA code,SUA name,Sum of shares in file
A.02.02.001,Garlic,01252,green garlic,0.518
A.02.03.005,Okra,01359.90,"Other fruits, n.e.c.",0.861
A.02.05.014,Grape leaves,01359.90,"Other fruits, n.e.c.",0.861
A.02.05.017,Oraches leaves,01359.90,"Other fruits, n.e.c.",0.861
A.02.07.008,Bamboo shoots,01359.90,"Other fruits, n.e.c.",0.861
A.02.07.009,Palm hearts,01359.90,"Other fruits, n.e.c.",0.861
A.02.15.001,Common mushrooms,01270,Mushrooms and truffles,2
A.02.16.002,Honey mushroom,01270,Mushrooms and truffles,2
A.02.16.003,Ceps,01270,Mushrooms and truffles,2
A.02.16.004,Truffles,01270,Mushrooms and truffles,2
A.02.16.005,Morels,01270,Mushrooms and truffles,2
A.02.16.006,Chanterelles,01270,Mushrooms and truffles,2
A.02.16.007,Saffron milk cap,01270,Mushrooms and truffles,2
A.04.04.004,Sesame seed (Sesamum indicum syn. S. orientale),01444,Sesame seed,0.677
A.04.04.007,Mustard seed (Brassica nigra),01442,Mustard seed,0.677
A.04.04.009,Pumpkin seeds,01449.90,"Other oil seeds, n.e.c.",0.677
A.04.04.013,Hempseed (Cannabis sativa),01929.02,"True hemp, raw or retted",0.677
A.04.05,Other oilseeds (including elsewhere non-listed seeds),01449.90,"Other oil seeds, n.e.c.",0.677
A.05.01.001,Grapefruits,01321,Pomelos and grapefruits,0.897
A.05.01.006,Pomelo (Citrus grandis),01321,Pomelos and grapefruits,0.897
A.05.02.004,Medlars,01359.90,"Other fruits, n.e.c.",0.861
A.05.02.005,Loquats,01359.90,"Other fruits, n.e.c.",0.861
A.05.04.004,Blackberries,01353.01,Raspberries,0.79
A.05.04.006,Raspberries (red and yellow),01353.01,Raspberries,0.79
A.05.04.008,Blueberries,01355.01,Blueberries,0.79
A.05.04.009,Cranberries (Vaccinium macrocarpon),01355.02,Cranberries,0.79
A.05.04.014,Azarole (mediteranean medlar) (Crataegus azarolus),01355.01,Blueberries,0.79
A.05.04.016,Cloudberries,01353.01,Raspberries,0.79
A.05.04.017,Loganberry (Rubus loganobaccus),01355.01,Blueberries,0.79
A.05.04.022,Bilberries,01355.01,Blueberries,0.79
A.05.04.026,Lingonberry (Vaccinium vitis-idaea),01355.01,Blueberries,0.79
A.05.05.002,Palm nuts (palmoil kernels),01491.02,palm kernels [only calories],0.677
A.05.05.003,Palmfruit (Elaeis guineensis),01491.01,oil palm fruit,2
A.05.06.005,Carambolas,01359.90,"Other fruits, n.e.c.",0.861
A.05.06.009,Litchis,01359.90,"Other fruits, n.e.c.",0.861
A.05.06.010,Passionfruits,01359.90,"Other fruits, n.e.c.",0.861
A.05.06.011,Prickly pears,01359.90,"Other fruits, n.e.c.",0.861
A.05.06.018,Granate apples,01359.90,"Other fruits, n.e.c.",0.861
A.05.06.019,Wild sweetsops,01359.90,"Other fruits, n.e.c.",0.861
A.05.06.022,Bread fruit (jackfruit) (Artocarpus altilis),01359.90,"Other fruits, n.e.c.",0.861
A.05.06.030,Chayote (Sechium edule),01359.90,"Other fruits, n.e.c.",0.861
A.05.06.034,Longans,01359.90,"Other fruits, n.e.c.",0.861
A.06.02.005,Pigeon fresh meat,21170.01,"meat of pigeons and other birds n.e.c., fresh, chilled or frozen",0.812
A.10.07,Honey,02910,Honey,0.696

IT

RPC Code,RPC Name,SUA code,SUA name,Sum of shares in file
A.02.05.022,Alfalfa sprouts,01709.90,other pulses n.e.c.,0.634
A.02.06.001,Beans (with pods) and similar-,01701,"beans, dry",0.634
A.02.10.002,"Tea (dried leaves and stalks, fermented or otherwise of Camellia sinensis), decaffeinated",01620,Tea leaves,0.876
A.02.15.001,Common mushrooms,01270,Mushrooms and truffles,1.819
A.02.15.002,Oyster mushroom (Pleurotus ostreatus),01270,Mushrooms and truffles,0.819
A.02.15.003,Shiitake,01270,Mushrooms and truffles,0.819
A.02.16.002,Honey mushroom,01270,Mushrooms and truffles,1.819
A.02.16.003,Ceps,01270,Mushrooms and truffles,1.819
A.02.16.004,Truffles,01270,Mushrooms and truffles,1.819
A.02.16.005,Morels,01270,Mushrooms and truffles,1.819
A.02.16.006,Chanterelles,01270,Mushrooms and truffles,1.819
A.02.16.007,Saffron milk cap,01270,Mushrooms and truffles,1.819
A.04.01.001,Beans (fresh seeds without pods) and similar-,01701,"Beans, dry",0.634
A.04.02.001,Beans (dry) and similar-,01701,"Beans, dry",0.634
A.04.02.008,Lima beans (dry),01701,"Beans, dry",0.634
A.04.02.009,Mung bean sprouts,01701,"Beans, dry",0.634
A.04.02.014,Carobs,01709.90,Other pulses n.e.c.,0.634
A.05.05.003,Palmfruit (Elaeis guineensis),01491.01,oil palm fruit,2
A.06.01.006,Equine muscle,21118.01,"horse meat, fresh or chilled",0.72
A.06.02.005,Pigeon fresh meat,21170.01,"meat of pigeons and other birds n.e.c., fresh, chilled or frozen",0.848
A.09.01.005,Duck eggs,0232,"eggs from other birds in shell, fresh, n.e.c.",0.857
A.09.01.006,Goose eggs,0232,"eggs from other birds in shell, fresh, n.e.c.",0.857
A.09.01.007,Quail eggs,0232,"eggs from other birds in shell, fresh, n.e.c.",0.857
A.10.07,Honey,02910,Honey,0.899

PL

RPC Code,RPC Name,SUA code,SUA name,Sum of shares in file
A.01.02.001,Common wheat grain,0111,wheat,0.119
A.01.02.001.003,Common wheat grain,0111,wheat,0.119
A.01.02.001.004,Durum wheat grain,0111,wheat,0.119
A.01.02.002.001,Barley grains,0115,barley,0.097
A.01.02.003,Maize grain,0112,maize (corn),0.072
A.01.02.005,Spelt grain,0111,wheat,0.119
A.01.02.008,Oat grain,0117,oats,0.021
A.02.01.001,Arrowroots,01599.10,"edible roots and tubers with high starch or inulin content, n.e.c., fresh",0.04
A.02.01.002,Beetroots,01599.10,"edible roots and tubers with high starch or inulin content, n.e.c., fresh",0.04
A.02.01.003,Carrots,01251,carrots and turnips,0.04
A.02.01.004,Celeriacs,01599.10,"edible roots and tubers with high starch or inulin content, n.e.c., fresh",0.04
A.02.01.005,Horseradish roots,01599.10,"edible roots and tubers with high starch or inulin content, n.e.c., fresh",0.04
A.02.01.006,Parsley roots,01599.10,"edible roots and tubers with high starch or inulin content, n.e.c., fresh",0.04
A.02.01.007,Parsnip roots,01599.10,"edible roots and tubers with high starch or inulin content, n.e.c., fresh",0.04
A.02.01.008,Radishes,01212,cabbages,0.01
A.02.01.010,Swedes,01212,cabbages,0.01
A.02.01.012,Turnips,01251,carrots and turnips,0.04
A.02.02.001,Garlic,01252,green garlic,0.836
A.02.02.002,Onions,01253.02,"onions and shallots, dry (excluding dehydrated)",0.108
A.02.02.003,Shallots,01253.02,"onions and shallots, dry (excluding dehydrated)",0.108
A.02.02.004,Spring onions,01253.02,"onions and shallots, dry (excluding dehydrated)",0.108
A.02.02.005,Chives,01253.02,"onions and shallots, dry (excluding dehydrated)",0.108
A.02.03.001,Tomatoes,01234,tomatoes,0.275
A.02.03.006,Cucumbers,01232,cucumbers and gherkins,0.079
A.02.03.007,Gherkins,01232,cucumbers and gherkins,0.079
A.02.03.008,Courgettes,01235,"pumpkins, squash and gourds",0.151
A.02.03.010,Pumpkins,01235,"pumpkins, squash and gourds",0.151
A.02.04.003,Brussels sprouts,01212,cabbages,0.01
A.02.04.004,Head cabbages,01212,cabbages,0.01
A.02.04.005,Chinese cabbages,01212,cabbages,0.01
A.02.04.006,Kales and similar-,01212,cabbages,0.01
A.02.04.007,Kohlrabies,01212,cabbages,0.01
A.02.05.001,Lamb's lettuces,01214,lettuce and chicory,0.555
A.02.05.002,Head lettuces,01214,lettuce and chicory,0.555
A.02.05.003,Crisp lettuces,01214,lettuce and chicory,0.555
A.02.05.004,Escaroles,01214,lettuce and chicory,0.555
A.02.05.005,Cresses,01212,cabbages,0.01
A.02.05.006,Land cresses,01212,cabbages,0.01
A.02.05.007,Roman rocket,01212,cabbages,0.01
A.02.05.013,Chards,01801,sugar beet,0.063
A.02.05.015,Watercresses,01212,cabbages,0.01
A.02.05.016,Witloofs,01214,lettuce and chicory,0.555
A.02.05.022,Alfalfa sprouts,01709.90,other pulses n.e.c.,0.155
A.02.06.001,Beans (with pods) and similar-,01701,"beans, dry",0.155
A.02.07.003,Celeries,01251,carrots and turnips,0.04
A.02.07.004,Florence fennels,01251,carrots and turnips,0.04
A.02.07.007,Rhubarbs,01801,sugar beet,0.063
A.02.08.001,Sugar beet (root) (Beta vulgaris),01801,sugar beet,0.063
A.02.10.002,"Tea (dried leaves and stalks, fermented or otherwise of Camellia sinensis), decaffeinated",01620,Tea leaves,0.891
A.02.15.001,Common mushrooms,01270,Mushrooms and truffles,2
A.02.16.002,Honey mushroom,01270,Mushrooms and truffles,2
A.02.16.003,Ceps,01270,Mushrooms and truffles,2
A.02.16.004,Truffles,01270,Mushrooms and truffles,2
A.02.16.005,Morels,01270,Mushrooms and truffles,2
A.02.16.006,Chanterelles,01270,Mushrooms and truffles,2
A.02.16.007,Saffron milk cap,01270,Mushrooms and truffles,2
A.03.01.001,New potatoes,01510,Potatoes,0.023
A.03.01.002,Main-crop potatoes,01510,Potatoes,0.023
A.04.01.001,Beans (fresh seeds without pods) and similar-,01701,"Beans, dry",0.155
A.04.01.002,Peas (without pods) and similar-,01242,"Peas, green",0.089
A.04.02.001,Beans (dry) and similar-,01701,"Beans, dry",0.155
A.04.02.008,Lima beans (dry),01701,"Beans, dry",0.155
A.04.02.009,Mung bean sprouts,01701,"Beans, dry",0.155
A.04.02.014,Carobs,01709.90,Other pulses n.e.c.,0.155
A.04.03.012,Walnuts,01376,"Walnuts, in shell",0.053
A.04.04.004,Sesame seed (Sesamum indicum syn. S. orientale),01444,Sesame seed,0.596
A.04.04.007,Mustard seed (Brassica nigra),01442,Mustard seed,0.596
A.04.04.009,Pumpkin seeds,01449.90,"Other oil seeds, n.e.c.",0.596
A.04.04.013,Hempseed (Cannabis sativa),01929.02,"True hemp, raw or retted",0.596
A.04.05,Other oilseeds (including elsewhere non-listed seeds),01449.90,"Other oil seeds, n.e.c.",0.596
A.05.01.001,Grapefruits,01321,Pomelos and grapefruits,0.812
A.05.01.006,Pomelo (Citrus grandis),01321,Pomelos and grapefruits,0.812
A.05.02.001,Apples,01341,Apples,0.055
A.05.02.002,Pears,01342.01,Pears,0.246
A.05.02.007,Sorb fruits,01341,Apples,0.055
A.05.02.009,Nashi pears,01342.01,Pears,0.246
A.05.03.001,Apricots,01343,Apricots,0.843
A.05.03.002,Plums,01346,Plums and sloes,0.321
A.05.03.003,Damsons,01346,Plums and sloes,0.321
A.05.03.004,Mirabelles,01346,Plums and sloes,0.321
A.05.03.005,Gages,01346,Plums and sloes,0.321
A.05.03.006,Cherries (sweet),01344.02,Cherries,0.019
A.05.03.011,Peaches and similar-,01345,Peaches and nectarines,0.898
A.05.04.003,Strawberries,01354,Strawberries,0.07
A.05.04.004,Blackberries,01353.01,Raspberries,0.094
A.05.04.006,Raspberries (red and yellow),01353.01,Raspberries,0.094
A.05.04.008,Blueberries,01355.01,Blueberries,0.094
A.05.04.009,Cranberries (Vaccinium macrocarpon),01355.02,Cranberries,0.094
A.05.04.014,Azarole (mediteranean medlar) (Crataegus azarolus),01355.01,Blueberries,0.094
A.05.04.016,Cloudberries,01353.01,Raspberries,0.094
A.05.04.017,Loganberry (Rubus loganobaccus),01355.01,Blueberries,0.094
A.05.04.022,Bilberries,01355.01,Blueberries,0.094
A.05.04.026,Lingonberry (Vaccinium vitis-idaea),01355.01,Blueberries,0.094
A.05.04.029,Ground cherries (physalis etc.),01234,Tomatoes,0.275
A.05.05.002,Palm nuts (palmoil kernels),01491.02,palm kernels [only calories],0.596
A.05.05.003,Palmfruit (Elaeis guineensis),01491.01,oil palm fruit,2
A.05.06.013,American persimmons,01234,Tomatoes,0.275
A.06.01.001,Bovine muscle,21111.02,"meat of cattle boneless, fresh or chilled",0.037
A.06.01.002,Calf fresh meat,21111.02,"meat of cattle boneless, fresh or chilled",0.037
A.06.01.003,Pig muscle,21113.02,"meat of pig boneless, fresh or chilled",0.263
A.06.01.004,Sheep muscle,21115,"meat of sheep, fresh or chilled",0.495
A.06.02.001,Chicken meat (Gallus domesticus),21121,"meat of chickens, fresh or chilled",0.016
A.06.02.002,Turkey fresh meat,21124,"meat of turkeys, fresh or chilled",0.11
A.06.02.006,Guinea-fowl fresh meat,21121,"meat of chickens, fresh or chilled",0.016
A.06.02.007,Ratites fresh meat,21170.02,"game meat, fresh, chilled or frozen",0.016
A.06.03.001,Wild boar fresh meat,21170.02,"game meat, fresh, chilled or frozen",0.016
A.06.03.002,"Deer, red fresh meat",21170.02,"game meat, fresh, chilled or frozen",0.016
A.06.03.003,European moose meat,21170.02,"game meat, fresh, chilled or frozen",0.016
A.06.03.004,Reindeer meat,21170.02,"game meat, fresh, chilled or frozen",0.016
A.06.04.001,Pheasant fresh meat,21170.02,"game meat, fresh, chilled or frozen",0.016
A.06.04.003,Mallard fresh meat,21170.02,"game meat, fresh, chilled or frozen",0.016
A.06.04.004,Partridge fresh meat,21170.02,"game meat, fresh, chilled or frozen",0.016
A.06.04.005,Quail fresh meat,21170.02,"game meat, fresh, chilled or frozen",0.016
A.08.01.001.001,"Cow milk, > 4% fat (inc. Channel Island milk)",02211,raw milk of cattle,0.104
A.08.01.001.002,Cow milk,02211,raw milk of cattle,0.104
A.08.01.007,Ass milk,02211,raw milk of cattle,0.104
A.09.01.001,Hen eggs,0231,"hen eggs in shell, fresh",0.044
A.10.07,Honey,02910,Honey,0.332

SE

RPC Code,RPC Name,SUA code,SUA name,Sum of shares in file
A.01.02.003,Maize grain,0112,maize (corn),0.791
A.02.02.001,Garlic,01252,green garlic,0.615
A.02.03.005,Okra,01359.90,"Other fruits, n.e.c.",0.78
A.02.05.014,Grape leaves,01359.90,"Other fruits, n.e.c.",0.78
A.02.05.017,Oraches leaves,01359.90,"Other fruits, n.e.c.",0.78
A.02.05.022,Alfalfa sprouts,01709.90,other pulses n.e.c.,0.682
A.02.06.001,Beans (with pods) and similar-,01701,"beans, dry",0.682
A.02.07.008,Bamboo shoots,01359.90,"Other fruits, n.e.c.",0.78
A.02.07.009,Palm hearts,01359.90,"Other fruits, n.e.c.",0.78
A.02.10.002,"Tea (dried leaves and stalks, fermented or otherwise of Camellia sinensis), decaffeinated",01620,Tea leaves,0.855
A.02.15.001,Common mushrooms,01270,Mushrooms and truffles,1.386
A.02.15.002,Oyster mushroom (Pleurotus ostreatus),01270,Mushrooms and truffles,0.386
A.02.15.003,Shiitake,01270,Mushrooms and truffles,0.386
A.02.16.002,Honey mushroom,01270,Mushrooms and truffles,1.386
A.02.16.003,Ceps,01270,Mushrooms and truffles,1.386
A.02.16.004,Truffles,01270,Mushrooms and truffles,1.386
A.02.16.005,Morels,01270,Mushrooms and truffles,1.386
A.02.16.006,Chanterelles,01270,Mushrooms and truffles,1.386
A.02.16.007,Saffron milk cap,01270,Mushrooms and truffles,1.386
A.04.01.001,Beans (fresh seeds without pods) and similar-,01701,"Beans, dry",0.682
A.04.02.001,Beans (dry) and similar-,01701,"Beans, dry",0.682
A.04.02.008,Lima beans (dry),01701,"Beans, dry",0.682
A.04.02.009,Mung bean sprouts,01701,"Beans, dry",0.682
A.04.02.014,Carobs,01709.90,Other pulses n.e.c.,0.682
A.04.04.004,Sesame seed (Sesamum indicum syn. S. orientale),01444,Sesame seed,0.761
A.04.04.007,Mustard seed (Brassica nigra),01442,Mustard seed,0.761
A.04.04.009,Pumpkin seeds,01449.90,"Other oil seeds, n.e.c.",0.761
A.04.04.013,Hempseed (Cannabis sativa),01929.02,"True hemp, raw or retted",0.761
A.04.05,Other oilseeds (including elsewhere non-listed seeds),01449.90,"Other oil seeds, n.e.c.",0.761
A.05.01.001,Grapefruits,01321,Pomelos and grapefruits,0.874
A.05.01.006,Pomelo (Citrus grandis),01321,Pomelos and grapefruits,0.874
A.05.02.001,Apples,01341,Apples,0.864
A.05.02.004,Medlars,01359.90,"Other fruits, n.e.c.",0.78
A.05.02.005,Loquats,01359.90,"Other fruits, n.e.c.",0.78
A.05.02.007,Sorb fruits,01341,Apples,0.864
A.05.05.002,Palm nuts (palmoil kernels),01491.02,palm kernels [only calories],0.761
A.05.05.003,Palmfruit (Elaeis guineensis),01491.01,oil palm fruit,2
A.05.06.005,Carambolas,01359.90,"Other fruits, n.e.c.",0.78
A.05.06.009,Litchis,01359.90,"Other fruits, n.e.c.",0.78
A.05.06.010,Passionfruits,01359.90,"Other fruits, n.e.c.",0.78
A.05.06.011,Prickly pears,01359.90,"Other fruits, n.e.c.",0.78
A.05.06.018,Granate apples,01359.90,"Other fruits, n.e.c.",0.78
A.05.06.019,Wild sweetsops,01359.90,"Other fruits, n.e.c.",0.78
A.05.06.022,Bread fruit (jackfruit) (Artocarpus altilis),01359.90,"Other fruits, n.e.c.",0.78
A.05.06.030,Chayote (Sechium edule),01359.90,"Other fruits, n.e.c.",0.78
A.05.06.034,Longans,01359.90,"Other fruits, n.e.c.",0.78
A.10.07,Honey,02910,Honey,0.786
I.07.01.006.002,Trouts,A029F,Trouts,2
I.07.01.015.001,Catfishes (freshwater),A027J,Catfishes (freshwater),2
I.07.01.017.002,seed,A02CG,Whiting,2
wanecek commented 1 month ago

There seems to be several issues contributing to this:

Minor Ingredients

Some of the sums adding up to more than 1 are due to added minor ingredients. Compare for example these rows from the SE.csv file above:

A.02.15.001,Common mushrooms,01270,Mushrooms and truffles,1.386
A.02.15.002,Oyster mushroom (Pleurotus ostreatus),01270,Mushrooms and truffles,0.386
A.02.15.003,Shiitake,01270,Mushrooms and truffles,0.386
A.02.16.002,Honey mushroom,01270,Mushrooms and truffles,1.386
A.02.16.003,Ceps,01270,Mushrooms and truffles,1.386
A.02.16.004,Truffles,01270,Mushrooms and truffles,1.386
A.02.16.005,Morels,01270,Mushrooms and truffles,1.386
A.02.16.006,Chanterelles,01270,Mushrooms and truffles,1.386
A.02.16.007,Saffron milk cap,01270,Mushrooms and truffles,1.386

those with a share of 1.386 have MI-XX rows added (with import of 100% from RoW) in addition to the shares originating from the Kastner files.

Invalid sums in Kastner script

Country-code collision

Multiple rows were invalid due to an error when generating the original files. Put simply, an error occurred where we 'renamed' countries (e.g. treating imports from Taiwan as those from china), where we, instead of adding imported amounts (e.g. china + Taiwan) managed to overwrite the values, as they (due to our aliasing) share the same country code. This caused part of the error with e.g. the mushrooms and truffles.

Fixed in the code in commit 9e84050ed55accb0b9085465deeb37819014ea34

Duplicate entries for RPC codes

Some long codes appeared twice in our rpc<->sua code matching file, e.g. for mung beans. This lead to that food-item appearing twice. This was fixed in f045e73081a5a3bb47b513180a16af3bdb03c6f7.

Invalid sums after rounding

After rounding and removing values, the sums sometimes did not add up to 1, but rather something close (e.g. 0.95 or 0.90), which is too far from the expected value. After commit f045e73081a5a3bb47b513180a16af3bdb03c6f7 we add adjust all shares after filtering out countries (e.g. those with imports of less than 1%) so that the sums add up to 1 again.