brightway-lca / brightway2-io

Importing and exporting for the Brightway LCA framework
BSD 3-Clause "New" or "Revised" License
26 stars 40 forks source link

SimaPro CSV importer doesn't fix broken uncertainty values #236

Open SamiraHuber opened 7 months ago

SamiraHuber commented 7 months ago

I tried to do the Monte Carlo analysis with the Agribalyse3.1 database, based on this notebook: https://github.com/brightway-lca/brightway2/blob/master/notebooks/Monte%20Carlo%20from%20Excel%20import.ipynb

mc = bw.MonteCarloLCA({activity:1},  ('IPCC 2013', 'climate change', 'GWP 100a'))
scores = [next(mc) for _ in range(10)]

The second line fails with the error: "Real, positive scale (sigma) values are required for lognormal uncertainties." The same happens in the acitivity browser. It works when using the Ecoinvent database.

I appreciate your help on how to fix it. Thanks a lot! Here are some more information:

My Agribalyse-Version: https://sustaincertcom-my.sharepoint.com/:x:/g/personal/gustave_coste_sustain-cert_com/ESwnbkR0VpRGrozFdZersVsBePolrNLmnVuCTeJbObSU4g?e=cTFO7Z&download=1

I took this from here: https://github.com/sc-gcoste/brightway-agribalyse3.1

My brightway versions: brightway2==2.4.2 bw-migrations==0.2 bw2analyzer==0.10 bw2data==1.8.1 bw2data==3.6.5 bw2io==0.8.8 bw2parameters==0.7

cmutel commented 7 months ago

@SamiraHuber Thanks for the bug report. I am moving this issue to the correct repo.

The SimaPro CSV importer should check the lognormal values and fix them where needed. We already do this in other importers.

In the meantime, you could do:

from numbers import Number

for ds in bw2data.Database("my database"):
    for exc in ds.exchanges():
            if (
                "uncertainty type" in exc
                and exc["uncertainty type"] in (2, 3)
                and (not isinstance(exc.get("scale"), Number) or exc["scale"] <= 0)
            ):
                print(f"Fixing exchange: {exc}")
                exc["previous uncertainty type"] = exc["uncertainty type"]
                exc["uncertainty type"] = 0
                exc["fixed nonpositive (log)normal scale"] = True
                exc["previous (log)normal scale value"] = exc.get("scale")
                exc["previous (log)normal loc value"] = exc.get("loc")
                exc["loc"] = exc["amount"]
                exc.save()
SamiraHuber commented 7 months ago

@cmutel Thanks a lot for the fast reply and moving it in the correct repo!

Sadly I get another error after the fixing of the values:

File [~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/distributions/base.py:135](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/distributions/base.py:135), in UncertaintyBase.validate(cls, params)
    [133](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/distributions/base.py:133) # Minimum <= Maximum
    [134](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/distributions/base.py:134) if (params['minimum'] >= params['maximum']).sum():
--> [135](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/distributions/base.py:135)     raise ImproperBoundsError

ImproperBoundsError:

Do you have an idea how I can handle this?

And the long log:

---------------------------------------------------------------------------
ImproperBoundsError                       Traceback (most recent call last)
[/Users/samirahuber/Code/lcai_api/backend/notebooks/monte_carlo.ipynb](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/monte_carlo.ipynb) Cell 11 line 3
      [1](vscode-notebook-cell:/Users/samirahuber/Code/lcai_api/backend/notebooks/monte_carlo.ipynb#X10sZmlsZQ%3D%3D?line=0) mc = bw.MonteCarloLCA({el: 1}, method)  
      [2](vscode-notebook-cell:/Users/samirahuber/Code/lcai_api/backend/notebooks/monte_carlo.ipynb#X10sZmlsZQ%3D%3D?line=1) #results = np.array([next(mc) for _ in range(10000)])
----> [3](vscode-notebook-cell:/Users/samirahuber/Code/lcai_api/backend/notebooks/monte_carlo.ipynb#X10sZmlsZQ%3D%3D?line=2) scores = [next(mc) for _ in range(10)]

[/Users/samirahuber/Code/lcai_api/backend/notebooks/monte_carlo.ipynb](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/monte_carlo.ipynb) Cell 11 line 3
      [1](vscode-notebook-cell:/Users/samirahuber/Code/lcai_api/backend/notebooks/monte_carlo.ipynb#X10sZmlsZQ%3D%3D?line=0) mc = bw.MonteCarloLCA({el: 1}, method)  
      [2](vscode-notebook-cell:/Users/samirahuber/Code/lcai_api/backend/notebooks/monte_carlo.ipynb#X10sZmlsZQ%3D%3D?line=1) #results = np.array([next(mc) for _ in range(10000)])
----> [3](vscode-notebook-cell:/Users/samirahuber/Code/lcai_api/backend/notebooks/monte_carlo.ipynb#X10sZmlsZQ%3D%3D?line=2) scores = [next(mc) for _ in range(10)]

File [~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/bw2calc/monte_carlo.py:101](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/bw2calc/monte_carlo.py:101), in MonteCarloLCA.__next__(self)
     [99](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/bw2calc/monte_carlo.py:99) def __next__(self):
    [100](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/bw2calc/monte_carlo.py:100)     if not hasattr(self, "tech_rng"):
--> [101](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/bw2calc/monte_carlo.py:101)         self.load_data()
    [102](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/bw2calc/monte_carlo.py:102)     self.rebuild_technosphere_matrix(self.tech_rng.next())
    [103](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/bw2calc/monte_carlo.py:103)     self.rebuild_biosphere_matrix(self.bio_rng.next())

File [~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/bw2calc/monte_carlo.py:88](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/bw2calc/monte_carlo.py:88), in MonteCarloLCA.load_data(self)
     [86](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/bw2calc/monte_carlo.py:86) def load_data(self):
     [87](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/bw2calc/monte_carlo.py:87)     self.load_lci_data()
---> [88](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/bw2calc/monte_carlo.py:88)     self.tech_rng = MCRandomNumberGenerator(self.tech_params, seed=self.seed)
     [89](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/bw2calc/monte_carlo.py:89)     self.bio_rng = MCRandomNumberGenerator(self.bio_params, seed=self.seed)
     [90](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/bw2calc/monte_carlo.py:90)     if self.lcia:

File [~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/random.py:146](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/random.py:146), in MCRandomNumberGenerator.__init__(self, params, maximum_iterations, seed, **kwargs)
    [144](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/random.py:144) self.choices = uncertainty_choices
    [145](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/random.py:145) self.random = np.random.RandomState(seed)
--> [146](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/random.py:146) self.verify_params()
    [147](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/random.py:147) self.ordering = np.argsort(self.params["uncertainty_type"])
    [148](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/random.py:148) self.reverse_ordering = np.argsort(self.ordering)

File [~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/random.py:174](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/random.py:174), in MCRandomNumberGenerator.verify_params(self)
    [172](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/random.py:172) mask = self.params[u'uncertainty_type'] == uncertainty_type.id
    [173](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/random.py:173) if mask.sum():
--> [174](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/random.py:174)     uncertainty_type.validate(self.params[mask])

File [~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/distributions/base.py:135](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/distributions/base.py:135), in UncertaintyBase.validate(cls, params)
    [133](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/distributions/base.py:133) # Minimum <= Maximum
    [134](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/distributions/base.py:134) if (params['minimum'] >= params['maximum']).sum():
--> [135](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/lcaiv2/lib/python3.11/site-packages/stats_arrays/distributions/base.py:135)     raise ImproperBoundsError

ImproperBoundsError:
cmutel commented 7 months ago

Can you please give me an idea of the number of exchanges fixed by the previous code?

We aren't seeing the kind of distribution here, just the fact that it supplies a min and max value, and that these are incorrect. Try this:

from numbers import Number

for ds in bw2data.Database("my database"):
    for exc in ds.exchanges():
        if (isinstance(exc.get("minimum"), Number) 
            and isinstance(exc.get("maximum"), Number)
            and exc["maximum"] <= exc["minimum"]):
            print(f"Fixing exchange min/max: {exc}")

            if exc.get("uncertainty type") is None:
                pass
            elif exc["uncertainty type"] in (4, 5):
                # Uniform and triangular
                exc["previous invalid uncertainty type"] = exc["uncertainty type"]
                exc["uncertainty type"] = 0
                exc["previous maximum value"] = exc.get("maximum")
                exc["previous minimum value"] = exc.get("minimum")
                exc["loc"] = exc["amount"]
            else:
                exc["previous maximum value"] = exc.get("maximum")
                exc["previous minimum value"] = exc.get("minimum")
            del exc["minimum"]
            del exc["maximum"]
            exc.save()
SamiraHuber commented 7 months ago

@cmutel With the first code snippet it updated 16 activities. I add also the list of the updated one. I try the new code you replyed in a second - thanks a lot already!

Fixing exchange: Exchange: 0.025641026 cubic meter 'Natural gas, high pressure {FR}| market for | Cut-off, S - Copied from Ecoinvent' (cubic meter, None, None) to 'Natural gas, burned in furnace >100kW of greenhouse' (megajoule, RER, None)>
Fixing exchange: Exchange: 1.11 megajoule 'Heat, waste' (megajoule, None, ('air', 'urban air close to ground')) to 'Natural gas, burned in furnace >100kW of greenhouse' (megajoule, RER, None)>
Fixing exchange: Exchange: 0.0234 kilogram 'Diesel {CH}| market for | Cut-off, S - Copied from Ecoinvent' (kilogram, None, None) to 'Diesel, burned in cogen 200kWe of greenhouse' (megajoule, CH, None)>
Fixing exchange: Exchange: 1.7838e-10 unit 'Municipal waste incineration facility {CH}| construction | Cut-off, S - Copied from Ecoinvent' (unit, None, None) to 'Disposal, grease tank residues, to municipal incineration, allocation price' (kilogram, CH, None)>
Fixing exchange: Exchange: 0.0071352 ton kilometer 'Municipal waste collection service by 21 metric ton lorry {CH}| processing | Cut-off, S - Copied from Ecoinvent' (ton kilometer, None, None) to 'Disposal, grease tank residues, to municipal incineration, allocation price' (kilogram, CH, None)>
Fixing exchange: Exchange: 1.7838e-10 unit 'Municipal waste incineration facility {CH}| construction | Cut-off, S - Copied from Ecoinvent' (unit, None, None) to 'Disposal, scrapings from pork products, to municipal incineration, allocation price' (kilogram, CH, None)>
Fixing exchange: Exchange: 0.0071352 ton kilometer 'Municipal waste collection service by 21 metric ton lorry {CH}| processing | Cut-off, S - Copied from Ecoinvent' (ton kilometer, None, None) to 'Disposal, scrapings from pork products, to municipal incineration, allocation price' (kilogram, CH, None)>
Fixing exchange: Exchange: 1.7838e-10 unit 'Municipal waste incineration facility {CH}| construction | Cut-off, S - Copied from Ecoinvent' (unit, None, None) to 'Disposal, animal byproducts (category 1), to municipal incineration, allocation price' (kilogram, CH, None)>
Fixing exchange: Exchange: 0.0071352 ton kilometer 'Municipal waste collection service by 21 metric ton lorry {CH}| processing | Cut-off, S - Copied from Ecoinvent' (ton kilometer, None, None) to 'Disposal, animal byproducts (category 1), to municipal incineration, allocation price' (kilogram, CH, None)>
Fixing exchange: Exchange: 2.5e-10 unit 'Municipal waste incineration facility {CH}| construction | Cut-off, S - Copied from Ecoinvent' (unit, None, None) to 'Disposal, ordinary industrial waste, 22.9% water, to municipal incineration' (kilogram, CH, None)>
Fixing exchange: Exchange: 2.5e-10 unit 'Municipal waste incineration facility {CH}| construction | Cut-off, S - Copied from Ecoinvent' (unit, None, None) to 'Disposal, plastic film, 0.4% water, to municipal incineration' (kilogram, CH, None)>
Fixing exchange: Exchange: 4.75 megajoule 'Heat, district or industrial, other than natural gas {RoW}| market for | Cut-off, S - Copied from Ecoinvent' (megajoule, None, None) to 'Drying of feed grain, processing' (litre, GB, None)>
Fixing exchange: Exchange: 5.56e-10 unit 'Sanitary landfill facility {CH}| construction | Cut-off, S - Copied from Ecoinvent' (unit, None, None) to 'Disposal, ordinary industrial waste, 22.9% water, to sanitary landfill' (kilogram, CH, None)>
Fixing exchange: Exchange: 2.1782e-07 kilometer 'Sewer grid, 5E9l/year, 110 km {CH}| construction | Cut-off, S - Copied from Ecoinvent' (kilometer, None, None) to 'Treatment, sewage, to wastewater treatment' (cubic meter, CH, None)>
Fixing exchange: Exchange: 5.6882e-09 unit 'Wastewater treatment facility, capacity 5E9l/year {CH}| construction | Cut-off, S - Copied from Ecoinvent' (unit, None, None) to 'Treatment, sewage, to wastewater treatment' (cubic meter, CH, None)>
Fixing exchange: Exchange: 4.75 megajoule 'Heat, district or industrial, other than natural gas {RoW}| market for | Cut-off, S - Copied from Ecoinvent' (megajoule, None, None) to 'Drying of feed grain, processing' (litre, UA, None)>
cmutel commented 7 months ago

@SamiraHuber Great, thanks. Please don't close this, I need to add a fix to the generic importer so no one else needs to deal with this :)

SamiraHuber commented 7 months ago

I don't know what I am doing wrong but after running the second code snippet, with updating the maximum and minimum, I again get the error I got in the beginning. Also when I run the first code you provided again, it still stays with the error.

The full error-log:

---------------------------------------------------------------------------
InvalidParamsError                        Traceback (most recent call last)
Cell In[23], [line 3](vscode-notebook-cell:?execution_count=23&line=3)
      [1](vscode-notebook-cell:?execution_count=23&line=1) mc = bw.MonteCarloLCA({el: 1}, method)  
      [2](vscode-notebook-cell:?execution_count=23&line=2) #results = np.array([next(mc) for _ in range(10000)])
----> [3](vscode-notebook-cell:?execution_count=23&line=3) scores = [next(mc) for _ in range(10)]

Cell In[23], [line 3](vscode-notebook-cell:?execution_count=23&line=3)
      [1](vscode-notebook-cell:?execution_count=23&line=1) mc = bw.MonteCarloLCA({el: 1}, method)  
      [2](vscode-notebook-cell:?execution_count=23&line=2) #results = np.array([next(mc) for _ in range(10000)])
----> [3](vscode-notebook-cell:?execution_count=23&line=3) scores = [next(mc) for _ in range(10)]

File [~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/bw2calc/monte_carlo.py:99](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/bw2calc/monte_carlo.py:99), in MonteCarloLCA.__next__(self)
     [97](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/bw2calc/monte_carlo.py:97) def __next__(self):
     [98](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/bw2calc/monte_carlo.py:98)     if not hasattr(self, "tech_rng"):
---> [99](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/bw2calc/monte_carlo.py:99)         self.load_data()
    [100](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/bw2calc/monte_carlo.py:100)     self.rebuild_technosphere_matrix(self.tech_rng.next())
    [101](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/bw2calc/monte_carlo.py:101)     self.rebuild_biosphere_matrix(self.bio_rng.next())

File [~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/bw2calc/monte_carlo.py:87](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/bw2calc/monte_carlo.py:87), in MonteCarloLCA.load_data(self)
     [85](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/bw2calc/monte_carlo.py:85) self.load_lci_data()
     [86](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/bw2calc/monte_carlo.py:86) self.tech_rng = MCRandomNumberGenerator(self.tech_params, seed=self.seed)
---> [87](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/bw2calc/monte_carlo.py:87) self.bio_rng = MCRandomNumberGenerator(self.bio_params, seed=self.seed)
     [88](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/bw2calc/monte_carlo.py:88) if self.lcia:
     [89](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/bw2calc/monte_carlo.py:89)     self.load_lcia_data()

File [~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/random.py:146](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/random.py:146), in MCRandomNumberGenerator.__init__(self, params, maximum_iterations, seed, **kwargs)
    [144](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/random.py:144) self.choices = uncertainty_choices
    [145](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/random.py:145) self.random = np.random.RandomState(seed)
--> [146](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/random.py:146) self.verify_params()
    [147](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/random.py:147) self.ordering = np.argsort(self.params["uncertainty_type"])
    [148](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/random.py:148) self.reverse_ordering = np.argsort(self.ordering)

File [~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/random.py:169](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/random.py:169), in MCRandomNumberGenerator.verify_params(self)
    [167](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/random.py:167) mask = self.params[u'uncertainty_type'] == uncertainty_type.id
    [168](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/random.py:168) if mask.sum():
--> [169](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/random.py:169)     uncertainty_type.validate(self.params[mask])

File [~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/distributions/normal.py:16](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/distributions/normal.py:16), in NormalUncertainty.validate(cls, params)
     [13](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/distributions/normal.py:13) @classmethod
     [14](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/distributions/normal.py:14) def validate(cls, params):
     [15](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/distributions/normal.py:15)     if np.isnan(params['scale']).sum() or (params['scale'] <= 0).sum():
---> [16](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/distributions/normal.py:16)         raise InvalidParamsError(
     [17](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/distributions/normal.py:17)             "Real, positive scale (sigma) values are required"
     [18](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/distributions/normal.py:18)             " for normal uncertainties."
     [19](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/distributions/normal.py:19)         )
     [20](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/distributions/normal.py:20)     if np.isnan(params['loc']).sum():
     [21](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/distributions/normal.py:21)         raise InvalidParamsError(
     [22](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/distributions/normal.py:22)             "Real loc (mu) values are required for normal uncertainties."
     [23](https://file+.vscode-resource.vscode-cdn.net/Users/samirahuber/Code/lcai_api/backend/notebooks/~/Code/lcai_api/bw_venv/lib/python3.11/site-packages/stats_arrays/distributions/normal.py:23)         )

InvalidParamsError: Real, positive scale (sigma) values are required for normal uncertainties.

The activities that it fixed:

Fixing exchange min/max: Exchange: -1.0 kilogram 'Fish residues {GLO}| fish residues, Recycled Content cut-off | Cut-off, S - Copied from Ecoinvent' (kilogram, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 2.0 kg*day 'Operation, reefer, freezing {GLO}| market for | Cut-off, S - Copied from Ecoinvent' (kg*day, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.2 kilogram 'Sodium chloride, powder {GLO}| market for | Cut-off, S - Copied from Ecoinvent' (kilogram, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.01 square meter 'Tin plated chromium steel sheet, 2 mm {GLO}| market for | Cut-off, S - Copied from Ecoinvent' (square meter, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.247 kilogram 'Tap water {RER}| market group for | Cut-off, S - Copied from Ecoinvent' (kilogram, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 4.181852251 kilogram 'Water, completely softened {RER}| market for water, completely softened | Cut-off, S - Copied from Ecoinvent' (kilogram, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 5.818147749 kilogram 'Water, completely softened {US}| market for water, completely softened | Cut-off, S - Copied from Ecoinvent' (kilogram, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.00225118 kilowatt hour 'Electricity, medium voltage {AR}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.003802678 kilowatt hour 'Electricity, medium voltage {AU}| market for | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.000152426 kilowatt hour 'Electricity, medium voltage {BO}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.010846169 kilowatt hour 'Electricity, medium voltage {BR}| market group for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.001207332 kilowatt hour 'Electricity, medium voltage {CL}| market for | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.001181148 kilowatt hour 'Electricity, medium voltage {CO}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.000186983 kilowatt hour 'Electricity, medium voltage {CR}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.000309932 kilowatt hour 'Electricity, medium voltage {CU}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 1.19e-05 kilowatt hour 'Electricity, medium voltage {CW}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.000257428 kilowatt hour 'Electricity, medium voltage {DO}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.00039014 kilowatt hour 'Electricity, medium voltage {EC}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.000185931 kilowatt hour 'Electricity, medium voltage {GT}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.000108607 kilowatt hour 'Electricity, medium voltage {HN}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 1.42e-05 kilowatt hour 'Electricity, medium voltage {HT}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 4.24e-05 kilowatt hour 'Electricity, medium voltage {JM}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.004408119 kilowatt hour 'Electricity, medium voltage {MX}| market for | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 6.31e-05 kilowatt hour 'Electricity, medium voltage {NI}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.00072919 kilowatt hour 'Electricity, medium voltage {NZ}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.000173891 kilowatt hour 'Electricity, medium voltage {PA}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.00031711 kilowatt hour 'Electricity, medium voltage {PR}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.000262003 kilowatt hour 'Electricity, medium voltage {PY}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.012241583 kilowatt hour 'Electricity, medium voltage {RAF}| market group for | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.178042871 kilowatt hour 'Electricity, medium voltage {RAS}| market group for | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.05643693 kilowatt hour 'Electricity, medium voltage {RER}| market group for | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.081764027 kilowatt hour 'Electricity, medium voltage {RNA}| market group for | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.016743349 kilowatt hour 'Electricity, medium voltage {RU}| market for | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 9.87e-05 kilowatt hour 'Electricity, medium voltage {SV}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.000150913 kilowatt hour 'Electricity, medium voltage {TT}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.000188604 kilowatt hour 'Electricity, medium voltage {UY}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.001431132 kilowatt hour 'Electricity, medium voltage {VE}| market for electricity, medium voltage | Cut-off, S - Copied from Ecoinvent' (kilowatt hour, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.070127943 megajoule 'Heat, district or industrial, other than natural gas {CA-QC}| market for | Cut-off, S - Copied from Ecoinvent' (megajoule, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 4.252872057 megajoule 'Heat, district or industrial, other than natural gas {RER}| market group for | Cut-off, S - Copied from Ecoinvent' (megajoule, None, None) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.00122 kilogram 'Nitrogen, atmospheric' (kilogram, None, ('water',)) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.00025 kilogram 'Phosphorus' (kilogram, None, ('water',)) to 'Fish canning, small fish {RoW}| fish canning, small fish | Cut-off, U - Adapted from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 3.06122448979592e-05 kilogram 'Carbon dioxide, liquid {RER}| market for | Cut-off, S - Copied from Ecoinvent' (kilogram, None, None) to 'Small pelagic fish, fresh {RoW}| tuna, capture by purse seiner and landing whole, frozen | Cut-off, U - Copied from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 3.06122448979592e-07 kilogram 'Chlorodifluoromethane {RER}| market for chlorodifluoromethane | Cut-off, S - Copied from Ecoinvent' (kilogram, None, None) to 'Small pelagic fish, fresh {RoW}| tuna, capture by purse seiner and landing whole, frozen | Cut-off, U - Copied from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.00559183673469388 kilogram 'Lubricating oil {RER}| market for lubricating oil | Cut-off, S - Copied from Ecoinvent' (kilogram, None, None) to 'Small pelagic fish, fresh {RoW}| tuna, capture by purse seiner and landing whole, frozen | Cut-off, U - Copied from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.250510204081633 kg*day 'Operation, reefer, freezing {GLO}| market for | Cut-off, S - Copied from Ecoinvent' (kg*day, None, None) to 'Small pelagic fish, fresh {RoW}| tuna, capture by purse seiner and landing whole, frozen | Cut-off, U - Copied from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 3.06122448979592e-05 kilogram 'Carbon dioxide, liquid {RoW}| market for | Cut-off, S - Copied from Ecoinvent' (kilogram, None, None) to 'Small pelagic fish, fresh {EC}| tuna, capture by purse seiner and landing whole, frozen | Cut-off, U - Copied from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 3.06122448979592e-07 kilogram 'Chlorodifluoromethane {RoW}| market for chlorodifluoromethane | Cut-off, S - Copied from Ecoinvent' (kilogram, None, None) to 'Small pelagic fish, fresh {EC}| tuna, capture by purse seiner and landing whole, frozen | Cut-off, U - Copied from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.00559183673469388 kilogram 'Lubricating oil {RoW}| market for lubricating oil | Cut-off, S - Copied from Ecoinvent' (kilogram, None, None) to 'Small pelagic fish, fresh {EC}| tuna, capture by purse seiner and landing whole, frozen | Cut-off, U - Copied from Ecoinvent' (kilogram, None, None)>
Fixing exchange min/max: Exchange: 0.250510204081633 kg*day 'Operation, reefer, freezing {GLO}| market for | Cut-off, S - Copied from Ecoinvent' (kg*day, None, None) to 'Small pelagic fish, fresh {EC}| tuna, capture by purse seiner and landing whole, frozen | Cut-off, U - Copied from Ecoinvent' (kilogram, None, None)>
cmutel commented 7 months ago

@SamiraHuber pretty sure we can figure out what is happening. First, let's look at the error message:

InvalidParamsError: Real, positive scale (sigma) values are required for normal uncertainties.

We made adjustments for the following uncertainty distributions: 2, 4, and 5 (Lognormal, Uniform, and Triangular). But we didn't fix normal distributions, and it's not surprising that they also have problems in this data.

The code snippet above has been updated to fix normal distributions as well.

SamiraHuber commented 7 months ago

Thanks a lot (also for helping out on a Sunday!), it is working!