LCA-ActivityBrowser / activity-browser

GUI for brightway2
GNU Lesser General Public License v3.0
144 stars 55 forks source link

Monte Carlo results seem off #1268

Closed MaximeAgez closed 5 months ago

MaximeAgez commented 5 months ago

Updating AB

What happened?

I ran multiple Monte Carlo analyses (100 iterations) on the "market for electricity, low voltage | electricity, low voltage | CA-QC | ecoinvent3.9.1 cut-off" process. The deterministic value for this process is 0.026651 kgCO2eq (using the IPCC2021 method). Yet, when I run Monte Carlo on this process alone (not modified or anything) the mean for the Monte Carlo is between 0.0293 and 0.0294 (did 4 times 100 iterations MC) and the median between 0.0285 and 0.0293. This is extremely weird that neither the mean or the median is remotely close from the deterministic result.

Similar issues observed on AB and on brightway2 itself (did not test brightway2.5).

Just to be sure I also tried with another process. Same issue with the process "market for 1-propanol | 1-propanol | GLO | ecoinvent3.9.1 cut-off" whose deterministic result is 6.6185kgCO2eq and MC results turn around 7.52kgCO2eq (mean) and 7.37kgCO2eq (median).

Relevant errors

No response

Operating system

Windows 11

Conda environment

activity-browser          2.9.7              pyhd8ed1ab_0    conda-forge
anyio                     4.3.0              pyhd8ed1ab_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
argon2-cffi               23.1.0             pyhd8ed1ab_0    conda-forge
argon2-cffi-bindings      21.2.0          py311ha68e1ae_4    conda-forge
arrow                     1.3.0              pyhd8ed1ab_0    conda-forge
asteval                   0.9.31             pyhd8ed1ab_0    conda-forge
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
astunparse                1.6.3              pyhd8ed1ab_0    conda-forge
async-lru                 2.0.4              pyhd8ed1ab_0    conda-forge
attrs                     23.2.0             pyh71513ae_0    conda-forge
babel                     2.14.0             pyhd8ed1ab_0    conda-forge
beautifulsoup4            4.12.3             pyha770c72_0    conda-forge
bleach                    6.1.0              pyhd8ed1ab_0    conda-forge
blinker                   1.7.0              pyhd8ed1ab_0    conda-forge
brightway2                2.4.4              pyhd8ed1ab_0    conda-forge
brotli                    1.1.0                hcfcfb64_1    conda-forge
brotli-bin                1.1.0                hcfcfb64_1    conda-forge
brotli-python             1.1.0           py311h12c1d0e_1    conda-forge
brotlicffi                1.1.0.0         py311h12c1d0e_1    conda-forge
bw2analyzer               0.10               pyhd8ed1ab_0    conda-forge
bw2calc                   1.8.2           py311h1ea47a8_0    conda-forge
bw2data                   3.6.6              pyhd8ed1ab_0    conda-forge
bw2io                     0.8.10             pyhd8ed1ab_0    conda-forge
bw2parameters             1.1.0              pyhd8ed1ab_0    conda-forge
bw_migrations             0.2                pyhd8ed1ab_0    conda-forge
bzip2                     1.0.8                hcfcfb64_5    conda-forge
ca-certificates           2024.2.2             h56e8100_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
certifi                   2024.2.2           pyhd8ed1ab_0    conda-forge
cffi                      1.16.0          py311ha68e1ae_0    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
click                     8.1.7           win_pyh7428d3b_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
comm                      0.2.1              pyhd8ed1ab_0    conda-forge
console_shortcut          0.1.1                         4
contourpy                 1.2.0           py311h005e61a_0    conda-forge
country-converter         1.2                      pypi_0    pypi
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
debugpy                   1.8.1           py311h12c1d0e_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
dill                      0.3.8              pyhd8ed1ab_0    conda-forge
docopt                    0.6.2                      py_1    conda-forge
eidl                      2.0.1              pyhd8ed1ab_0    conda-forge
eight                     1.0.1              pyhd8ed1ab_4    conda-forge
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
et_xmlfile                1.1.0              pyhd8ed1ab_0    conda-forge
exceptiongroup            1.2.0              pyhd8ed1ab_2    conda-forge
executing                 2.0.1              pyhd8ed1ab_0    conda-forge
fasteners                 0.17.3             pyhd8ed1ab_0    conda-forge
flask                     3.0.2              pyhd8ed1ab_0    conda-forge
fonttools                 4.49.0          py311ha68e1ae_0    conda-forge
fqdn                      1.5.1              pyhd8ed1ab_0    conda-forge
freetype                  2.12.1               hdaf720e_2    conda-forge
fsspec                    2024.3.1                 pypi_0    pypi
future                    1.0.0              pyhd8ed1ab_0    conda-forge
gettext                   0.21.1               h5728263_0    conda-forge
glib                      2.78.4               h12be248_0    conda-forge
glib-tools                2.78.4               h12be248_0    conda-forge
gst-plugins-base          1.22.9               h001b923_0    conda-forge
gstreamer                 1.22.9               hb4038d2_0    conda-forge
h11                       0.14.0             pyhd8ed1ab_0    conda-forge
h2                        4.1.0              pyhd8ed1ab_0    conda-forge
hpack                     4.0.0              pyh9f0ad1d_0    conda-forge
httpcore                  1.0.4              pyhd8ed1ab_0    conda-forge
httpx                     0.27.0             pyhd8ed1ab_0    conda-forge
hyperframe                6.0.1              pyhd8ed1ab_0    conda-forge
icu                       73.2                 h63175ca_0    conda-forge
idna                      3.6                pyhd8ed1ab_0    conda-forge
importlib-metadata        7.0.1              pyha770c72_0    conda-forge
importlib_metadata        7.0.1                hd8ed1ab_0    conda-forge
importlib_resources       6.1.1              pyhd8ed1ab_0    conda-forge
inflate64                 1.0.0           py311ha68e1ae_1    conda-forge
intel-openmp              2024.0.0         h57928b3_49841    conda-forge
ipykernel                 6.29.2             pyha63f2e9_0    conda-forge
ipython                   8.22.1             pyh7428d3b_0    conda-forge
ipywidgets                8.1.2              pyhd8ed1ab_0    conda-forge
isoduration               20.11.0            pyhd8ed1ab_0    conda-forge
itsdangerous              2.1.2              pyhd8ed1ab_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
jinja2                    3.1.3              pyhd8ed1ab_0    conda-forge
json5                     0.9.17             pyhd8ed1ab_0    conda-forge
jsonpointer               2.4             py311h1ea47a8_3    conda-forge
jsonschema                4.21.1             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2023.12.1          pyhd8ed1ab_0    conda-forge
jsonschema-with-format-nongpl 4.21.1             pyhd8ed1ab_0    conda-forge
jupyter                   1.0.0             pyhd8ed1ab_10    conda-forge
jupyter-lsp               2.2.2              pyhd8ed1ab_0    conda-forge
jupyter_client            8.6.0              pyhd8ed1ab_0    conda-forge
jupyter_console           6.6.3              pyhd8ed1ab_0    conda-forge
jupyter_core              5.7.1           py311h1ea47a8_0    conda-forge
jupyter_events            0.9.0              pyhd8ed1ab_0    conda-forge
jupyter_server            2.12.5             pyhd8ed1ab_0    conda-forge
jupyter_server_terminals  0.5.2              pyhd8ed1ab_0    conda-forge
jupyterlab                4.1.2              pyhd8ed1ab_0    conda-forge
jupyterlab_pygments       0.3.0              pyhd8ed1ab_1    conda-forge
jupyterlab_server         2.25.3             pyhd8ed1ab_0    conda-forge
jupyterlab_widgets        3.0.10             pyhd8ed1ab_0    conda-forge
kiwisolver                1.4.5           py311h005e61a_1    conda-forge
krb5                      1.21.2               heb0366b_0    conda-forge
lcms2                     2.16                 h67d730c_0    conda-forge
lerc                      4.0.0                h63175ca_0    conda-forge
libblas                   3.9.0              21_win64_mkl    conda-forge
libbrotlicommon           1.1.0                hcfcfb64_1    conda-forge
libbrotlidec              1.1.0                hcfcfb64_1    conda-forge
libbrotlienc              1.1.0                hcfcfb64_1    conda-forge
libcblas                  3.9.0              21_win64_mkl    conda-forge
libclang                  15.0.7          default_h77d9078_3    conda-forge
libclang13                15.0.7          default_h77d9078_3    conda-forge
libdeflate                1.19                 hcfcfb64_0    conda-forge
libexpat                  2.5.0                h63175ca_1    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libglib                   2.78.4               h16e383f_0    conda-forge
libhwloc                  2.9.1                h51c2c0f_0    conda-forge
libiconv                  1.17                 hcfcfb64_2    conda-forge
libjpeg-turbo             3.0.0                hcfcfb64_1    conda-forge
liblapack                 3.9.0              21_win64_mkl    conda-forge
libogg                    1.3.4                h8ffe710_1    conda-forge
libpng                    1.6.43               h19919ed_0    conda-forge
libsodium                 1.0.18               h8d14728_1    conda-forge
libsqlite                 3.45.1               hcfcfb64_0    conda-forge
libtiff                   4.6.0                h6e2ebb7_2    conda-forge
libvorbis                 1.3.7                h0e60522_0    conda-forge
libwebp                   1.3.2                hcfcfb64_1    conda-forge
libwebp-base              1.3.2                hcfcfb64_0    conda-forge
libxcb                    1.15                 hcd874cb_0    conda-forge
libxml2                   2.10.4               hc3477c8_0    conda-forge
libxslt                   1.1.37               h0192164_0    conda-forge
libzlib                   1.2.13               hcfcfb64_5    conda-forge
lxml                      4.9.2           py311h5942461_0    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
markupsafe                2.1.5           py311ha68e1ae_0    conda-forge
matplotlib-base           3.8.3           py311h6e989c2_0    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
mistune                   3.0.2              pyhd8ed1ab_0    conda-forge
mkl                       2024.0.0         h66d3029_49657    conda-forge
mrio_common_metadata      0.2.1              pyhd8ed1ab_0    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
multiprocess              0.70.16         py311ha68e1ae_0    conda-forge
multivolumefile           0.2.3              pyhd8ed1ab_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
nbclient                  0.8.0              pyhd8ed1ab_0    conda-forge
nbconvert                 7.16.1             pyhd8ed1ab_0    conda-forge
nbconvert-core            7.16.1             pyhd8ed1ab_0    conda-forge
nbconvert-pandoc          7.16.1             pyhd8ed1ab_0    conda-forge
nbformat                  5.9.2              pyhd8ed1ab_0    conda-forge
nest-asyncio              1.6.0              pyhd8ed1ab_0    conda-forge
networkx                  3.2.1              pyhd8ed1ab_0    conda-forge
notebook                  7.1.0              pyhd8ed1ab_0    conda-forge
notebook-shim             0.2.4              pyhd8ed1ab_0    conda-forge
numpy                     1.23.5          py311h95d790f_0    conda-forge
openjpeg                  2.5.0                h3d672ee_3    conda-forge
openpyxl                  3.1.2           py311ha68e1ae_1    conda-forge
openssl                   3.2.1                hcfcfb64_0    conda-forge
overrides                 7.7.0              pyhd8ed1ab_0    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pandas                    2.1.4           py311hf63dbb6_0    conda-forge
pandoc                    3.1.12.1             h57928b3_0    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pathos                    0.3.2              pyhd8ed1ab_1    conda-forge
patsy                     0.5.6              pyhd8ed1ab_0    conda-forge
pcre2                     10.42                h17e33f8_0    conda-forge
peewee                    3.17.1          py311had142b2_0    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.2.0          py311h4dd8a23_0    conda-forge
pint                      0.21               pyhd8ed1ab_0    conda-forge
pip                       24.0               pyhd8ed1ab_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              4.2.0              pyhd8ed1ab_0    conda-forge
pox                       0.3.4              pyhd8ed1ab_0    conda-forge
ppft                      1.7.6.8            pyhd8ed1ab_0    conda-forge
prometheus_client         0.20.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.42             pyha770c72_0    conda-forge
prompt_toolkit            3.0.42               hd8ed1ab_0    conda-forge
psutil                    5.9.8           py311ha68e1ae_0    conda-forge
pthread-stubs             0.4               hcd874cb_1001    conda-forge
pthreads-win32            2.9.1                hfa6e2cd_3    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
py7zr                     0.20.8             pyhd8ed1ab_1    conda-forge
pybcj                     1.0.2           py311ha68e1ae_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pycryptodomex             3.19.0          py311ha68e1ae_1    conda-forge
pygments                  2.17.2             pyhd8ed1ab_0    conda-forge
pypardiso                 0.4.3              pyhd8ed1ab_0    conda-forge
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pyperclip                 1.8.2              pyhd8ed1ab_2    conda-forge
pyppmd                    1.1.0           py311h12c1d0e_0    conda-forge
pyprind                   2.11.2          py311h1ea47a8_1005    conda-forge
pyside2                   5.15.8          py311h2411a93_2    conda-forge
pysocks                   1.7.1              pyh0701188_6    conda-forge
python                    3.11.8          h2628c8c_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.19.1             pyhd8ed1ab_0    conda-forge
python-json-logger        2.0.7              pyhd8ed1ab_0    conda-forge
python-tzdata             2024.1             pyhd8ed1ab_0    conda-forge
python_abi                3.11                    4_cp311    conda-forge
pytz                      2024.1             pyhd8ed1ab_0    conda-forge
pywin32                   306             py311h12c1d0e_2    conda-forge
pywinpty                  2.0.12          py311h12c1d0e_0    conda-forge
pyxlsb                    1.0.10             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0.1           py311ha68e1ae_1    conda-forge
pyzmq                     25.1.2          py311h9250fbb_0    conda-forge
pyzstd                    0.15.9          py311he5d195f_1    conda-forge
qt-main                   5.15.8              h9e85ed6_19    conda-forge
qt-webengine              5.15.8               h4bf5c4e_4    conda-forge
qtconsole-base            5.5.1              pyha770c72_0    conda-forge
qtpy                      2.4.1              pyhd8ed1ab_0    conda-forge
referencing               0.33.0             pyhd8ed1ab_0    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
rfc3339-validator         0.1.4              pyhd8ed1ab_0    conda-forge
rfc3986-validator         0.1.1              pyh9f0ad1d_0    conda-forge
rpds-py                   0.18.0          py311hc37eb10_0    conda-forge
salib                     1.4.8              pyhd8ed1ab_0    conda-forge
scipy                     1.12.0          py311h0b4df5a_2    conda-forge
seaborn                   0.13.2               hd8ed1ab_0    conda-forge
seaborn-base              0.13.2             pyhd8ed1ab_0    conda-forge
send2trash                1.8.2              pyh08f2357_0    conda-forge
setuptools                69.1.1             pyhd8ed1ab_0    conda-forge
setuptools-scm            8.0.4              pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sniffio                   1.3.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
stats_arrays              0.6.6              pyhd8ed1ab_0    conda-forge
statsmodels               0.14.1          py311h59ca53f_0    conda-forge
tabulate                  0.9.0              pyhd8ed1ab_1    conda-forge
tbb                       2021.9.0             h91493d7_0    conda-forge
terminado                 0.18.0             pyh5737063_0    conda-forge
texttable                 1.7.0              pyhd8ed1ab_0    conda-forge
tinycss2                  1.2.1              pyhd8ed1ab_0    conda-forge
tk                        8.6.13               h5226925_1    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tornado                   6.4             py311ha68e1ae_0    conda-forge
tqdm                      4.66.2             pyhd8ed1ab_0    conda-forge
traitlets                 5.14.1             pyhd8ed1ab_0    conda-forge
types-python-dateutil     2.8.19.20240106    pyhd8ed1ab_0    conda-forge
typing-extensions         4.9.0                hd8ed1ab_0    conda-forge
typing_extensions         4.9.0              pyha770c72_0    conda-forge
typing_utils              0.1.0              pyhd8ed1ab_0    conda-forge
tzdata                    2024a                h0c530f3_0    conda-forge
ucrt                      10.0.22621.0         h57928b3_0    conda-forge
unicodecsv                0.14.1             pyhd8ed1ab_2    conda-forge
unidecode                 1.3.8              pyhd8ed1ab_0    conda-forge
uri-template              1.3.0              pyhd8ed1ab_0    conda-forge
urllib3                   2.2.1              pyhd8ed1ab_0    conda-forge
vc                        14.3                hcf57466_18    conda-forge
vc14_runtime              14.38.33130         h82b7239_18    conda-forge
voluptuous                0.14.2             pyhd8ed1ab_0    conda-forge
vs2015_runtime            14.38.33130         hcb4865c_18    conda-forge
wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
webcolors                 1.13               pyhd8ed1ab_0    conda-forge
webencodings              0.5.1              pyhd8ed1ab_2    conda-forge
websocket-client          1.7.0              pyhd8ed1ab_0    conda-forge
werkzeug                  3.0.1              pyhd8ed1ab_0    conda-forge
wheel                     0.42.0             pyhd8ed1ab_0    conda-forge
whoosh                    2.7.4           py311h1ea47a8_8    conda-forge
widgetsnbextension        4.0.10             pyhd8ed1ab_0    conda-forge
win_inet_pton             1.1.0              pyhd8ed1ab_6    conda-forge
winpty                    0.4.3                         4    conda-forge
wrapt                     1.16.0          py311ha68e1ae_0    conda-forge
xlrd                      2.0.1              pyhd8ed1ab_3    conda-forge
xlsxwriter                3.1.9              pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.11               hcd874cb_0    conda-forge
xorg-libxdmcp             1.1.3                hcd874cb_0    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
yaml                      0.2.5                h8ffe710_2    conda-forge
zeromq                    4.3.5                h63175ca_0    conda-forge
zipfile-deflate64         0.2.0           py311ha68e1ae_4    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zstd                      1.5.5                h12be248_0    conda-forge
marc-vdm commented 5 months ago

Similar issues observed on AB and on brightway2 itself

@cmutel, any thoughts?

romainsacchi commented 5 months ago

https://stackoverflow.com/questions/72660400/variation-between-montecarlo-mean-score-and-static-lca-score

marc-vdm commented 5 months ago

@romainsacchi thanks for answering!

I will close this for now. If there are more questions regarding MC results, I suggest we open a new issue on brightway-lca/brightway2-calc.