LCA-ActivityBrowser / activity-browser

GUI for brightway2
GNU Lesser General Public License v3.0
144 stars 57 forks source link

ecoinvent importer failing with `bw2io=0.8.10` #1137

Closed marc-vdm closed 5 months ago

marc-vdm commented 10 months ago

I recently updated the AB on my computer by creating a new environment and installing the AB as new two days ago. I am now trying to import ecoinvent 3.9.1 in one project. I created a new project and imported biosphere flows.

I am now trying to import ecoinvent 3.9.1. I tried several ways:

  • Import of remote data with ecoinvent login,
  • Import local data with the ecoSpold2.7z file downloaded on the ecoinvent website,
  • Import local data with the 7z file found in the EcoInventDownLoader folder on my computer (generated by previous imports I guess). I have been trying several times per day the last two days but the AB crashes every time. I do not get any error on the console or any information on the log file. Sometimes it crashes during the extraction of datasets, sometimes after a few brightway strategies are implemented.

Did somebody experience the same issue? How could I try to resolve it?

Thank you in advance for your help, Best regards, Adeline

Originally posted by @AdelineJerome in https://github.com/LCA-ActivityBrowser/activity-browser/discussions/1135

Eventual solution was installing AB 2.8.0 which uses bw2io=0.8.9 conda create -n ab28 -c conda-forge activity-browser=2.8.0

Current package list ``` # Name Version Build Channel activity-browser 2.9.2 pyhd8ed1ab_0 conda-forge appdirs 1.4.4 pyh9f0ad1d_0 conda-forge arrow 1.3.0 pyhd8ed1ab_0 conda-forge asteval 0.9.31 pyhd8ed1ab_0 conda-forge astunparse 1.6.3 pyhd8ed1ab_0 conda-forge blinker 1.7.0 pyhd8ed1ab_0 conda-forge brightway2 2.4.4 pyhd8ed1ab_0 conda-forge brotli 1.1.0 hcfcfb64_1 conda-forge brotli-bin 1.1.0 hcfcfb64_1 conda-forge brotli-python 1.1.0 py39h99910a6_1 conda-forge brotlicffi 1.1.0.0 py39h99910a6_1 conda-forge bw2analyzer 0.10 pyhd8ed1ab_0 conda-forge bw2calc 1.8.2 py39hcbf5309_0 conda-forge bw2data 3.6.6 pyhd8ed1ab_0 conda-forge bw2io 0.8.10 pyhd8ed1ab_0 conda-forge bw2parameters 1.1.0 pyhd8ed1ab_0 conda-forge bw_migrations 0.2 pyhd8ed1ab_0 conda-forge bzip2 1.0.8 hcfcfb64_5 conda-forge ca-certificates 2023.11.17 h56e8100_0 conda-forge certifi 2023.11.17 pyhd8ed1ab_0 conda-forge cffi 1.16.0 py39ha55989b_0 conda-forge charset-normalizer 3.3.2 pyhd8ed1ab_0 conda-forge click 8.1.7 win_pyh7428d3b_0 conda-forge colorama 0.4.6 pyhd8ed1ab_0 conda-forge contourpy 1.2.0 py39h1f6ef14_0 conda-forge cycler 0.12.1 pyhd8ed1ab_0 conda-forge dill 0.3.7 pyhd8ed1ab_0 conda-forge docopt 0.6.2 py_1 conda-forge eidl 2.0.1 pyhd8ed1ab_0 conda-forge eight 1.0.1 pyhd8ed1ab_4 conda-forge et_xmlfile 1.1.0 pyhd8ed1ab_0 conda-forge fasteners 0.17.3 pyhd8ed1ab_0 conda-forge flask 3.0.0 pyhd8ed1ab_0 conda-forge fonttools 4.44.3 py39ha55989b_0 conda-forge freetype 2.12.1 hdaf720e_2 conda-forge future 0.18.3 pyhd8ed1ab_0 conda-forge gettext 0.21.1 h5728263_0 conda-forge glib 2.78.1 h12be248_1 conda-forge glib-tools 2.78.1 h12be248_1 conda-forge gst-plugins-base 1.22.7 h001b923_0 conda-forge gstreamer 1.22.7 hb4038d2_0 conda-forge icu 73.2 h63175ca_0 conda-forge idna 3.4 pyhd8ed1ab_0 conda-forge importlib-metadata 6.8.0 pyha770c72_0 conda-forge importlib-resources 6.1.1 pyhd8ed1ab_0 conda-forge importlib_metadata 6.8.0 hd8ed1ab_0 conda-forge importlib_resources 6.1.1 pyhd8ed1ab_0 conda-forge inflate64 1.0.0 py39ha55989b_0 conda-forge intel-openmp 2023.2.0 h57928b3_50497 conda-forge itsdangerous 2.1.2 pyhd8ed1ab_0 conda-forge jinja2 3.1.2 pyhd8ed1ab_1 conda-forge kiwisolver 1.4.5 py39h1f6ef14_1 conda-forge krb5 1.21.2 heb0366b_0 conda-forge lcms2 2.15 h67d730c_3 conda-forge lerc 4.0.0 h63175ca_0 conda-forge libblas 3.9.0 19_win64_mkl conda-forge libbrotlicommon 1.1.0 hcfcfb64_1 conda-forge libbrotlidec 1.1.0 hcfcfb64_1 conda-forge libbrotlienc 1.1.0 hcfcfb64_1 conda-forge libcblas 3.9.0 19_win64_mkl conda-forge libclang 15.0.7 default_h77d9078_3 conda-forge libclang13 15.0.7 default_h77d9078_3 conda-forge libdeflate 1.19 hcfcfb64_0 conda-forge libffi 3.4.4 hd77b12b_0 libglib 2.78.1 h16e383f_1 conda-forge libhwloc 2.9.1 h51c2c0f_0 conda-forge libiconv 1.17 h8ffe710_0 conda-forge libjpeg-turbo 3.0.0 hcfcfb64_1 conda-forge liblapack 3.9.0 19_win64_mkl conda-forge libogg 1.3.5 h2bbff1b_1 libpng 1.6.39 h19919ed_0 conda-forge libsqlite 3.44.0 hcfcfb64_0 conda-forge libtiff 4.6.0 h6e2ebb7_2 conda-forge libvorbis 1.3.7 h0e60522_0 conda-forge libwebp-base 1.3.2 hcfcfb64_0 conda-forge libxcb 1.15 hcd874cb_0 conda-forge libxml2 2.10.4 hc3477c8_0 conda-forge libxslt 1.1.37 h0192164_0 conda-forge libzlib 1.2.13 hcfcfb64_5 conda-forge lxml 4.9.3 py39h09808a7_0 m2w64-gcc-libgfortran 5.3.0 6 conda-forge m2w64-gcc-libs 5.3.0 7 conda-forge m2w64-gcc-libs-core 5.3.0 7 conda-forge m2w64-gmp 6.1.0 2 conda-forge m2w64-libwinpthread-git 5.0.0.4634.697f757 2 conda-forge markupsafe 2.1.3 py39ha55989b_1 conda-forge matplotlib-base 3.8.1 py39hf19769e_0 conda-forge mkl 2023.2.0 h6a75c08_50496 conda-forge mrio_common_metadata 0.2.1 pyhd8ed1ab_0 conda-forge msys2-conda-epoch 20160418 1 conda-forge multiprocess 0.70.15 py39ha55989b_1 conda-forge multivolumefile 0.2.3 pyhd8ed1ab_0 conda-forge munkres 1.1.4 pyh9f0ad1d_0 conda-forge networkx 3.2.1 pyhd8ed1ab_0 conda-forge numpy 1.23.5 py39hbccbffa_0 conda-forge openjpeg 2.5.0 h3d672ee_3 conda-forge openpyxl 3.1.2 py39ha55989b_1 conda-forge openssl 3.1.4 hcfcfb64_0 conda-forge packaging 23.2 pyhd8ed1ab_0 conda-forge pandas 2.1.3 py39h32e6231_0 conda-forge pathos 0.3.1 pyhd8ed1ab_0 conda-forge patsy 0.5.3 pyhd8ed1ab_0 conda-forge pcre2 10.42 h17e33f8_0 conda-forge peewee 3.17.0 py39habf9222_0 conda-forge pillow 10.1.0 py39h368b509_0 conda-forge pint 0.21 pyhd8ed1ab_0 conda-forge pip 23.3.1 pyhd8ed1ab_0 conda-forge pox 0.3.3 pyhd8ed1ab_0 conda-forge ppft 1.7.6.7 pyhd8ed1ab_0 conda-forge psutil 5.9.5 py39ha55989b_1 conda-forge pthread-stubs 0.4 hcd874cb_1001 conda-forge pthreads-win32 2.9.1 hfa6e2cd_3 conda-forge py7zr 0.20.8 pyhd8ed1ab_0 conda-forge pybcj 1.0.2 py39ha55989b_0 conda-forge pycparser 2.21 pyhd8ed1ab_0 conda-forge pycryptodomex 3.19.0 py39ha55989b_1 conda-forge pypardiso 0.4.3 pyhd8ed1ab_0 conda-forge pyparsing 3.1.1 pyhd8ed1ab_0 conda-forge pyperclip 1.8.2 pyhd8ed1ab_2 conda-forge pyppmd 1.0.0 py39h99910a6_2 conda-forge pyprind 2.11.2 py39hcbf5309_1005 conda-forge pyside2 5.15.8 py39h56cbfbc_2 conda-forge pysocks 1.7.1 py39hcbf5309_5 conda-forge python 3.9.18 h4de0772_0_cpython conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python-tzdata 2023.3 pyhd8ed1ab_0 conda-forge python_abi 3.9 4_cp39 conda-forge pytz 2023.3.post1 pyhd8ed1ab_0 conda-forge pyxlsb 1.0.10 pyhd8ed1ab_0 conda-forge pyzstd 0.15.9 py39h95af829_1 conda-forge qt-main 5.15.8 h9e85ed6_17 conda-forge qt-webengine 5.15.9 h5bd16bc_7 requests 2.31.0 pyhd8ed1ab_0 conda-forge salib 1.4.7 pyhd8ed1ab_0 conda-forge scipy 1.11.3 py39hddb5d58_1 conda-forge seaborn 0.13.0 hd8ed1ab_0 conda-forge seaborn-base 0.13.0 pyhd8ed1ab_0 conda-forge setuptools 68.2.2 pyhd8ed1ab_0 conda-forge setuptools-scm 8.0.4 pyhd8ed1ab_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge stats_arrays 0.6.6 pyhd8ed1ab_0 conda-forge statsmodels 0.14.0 py39hd88c2e4_2 conda-forge tabulate 0.9.0 pyhd8ed1ab_1 conda-forge tbb 2021.9.0 h91493d7_0 conda-forge texttable 1.7.0 pyhd8ed1ab_0 conda-forge tk 8.6.13 h5226925_1 conda-forge tomli 2.0.1 pyhd8ed1ab_0 conda-forge tqdm 4.66.1 pyhd8ed1ab_0 conda-forge types-python-dateutil 2.8.19.14 pyhd8ed1ab_0 conda-forge typing-extensions 4.8.0 hd8ed1ab_0 conda-forge typing_extensions 4.8.0 pyha770c72_0 conda-forge tzdata 2023c h71feb2d_0 conda-forge ucrt 10.0.22621.0 h57928b3_0 conda-forge unicodecsv 0.14.1 py_1 conda-forge unicodedata2 15.1.0 py39ha55989b_0 conda-forge unidecode 1.3.7 pyhd8ed1ab_0 conda-forge urllib3 2.1.0 pyhd8ed1ab_0 conda-forge vc 14.3 h64f974e_17 conda-forge vc14_runtime 14.36.32532 hdcecf7f_17 conda-forge voluptuous 0.14.1 pyhd8ed1ab_0 conda-forge vs2015_runtime 14.36.32532 h05e6639_17 conda-forge werkzeug 3.0.1 pyhd8ed1ab_0 conda-forge wheel 0.41.3 pyhd8ed1ab_0 conda-forge whoosh 2.7.4 py39hcbf5309_8 conda-forge win_inet_pton 1.1.0 py39hcbf5309_5 conda-forge wrapt 1.16.0 py39ha55989b_0 conda-forge xlrd 2.0.1 pyhd8ed1ab_3 conda-forge xlsxwriter 3.1.9 pyhd8ed1ab_0 conda-forge xorg-libxau 1.0.11 hcd874cb_0 conda-forge xorg-libxdmcp 1.1.3 hcd874cb_0 conda-forge xz 5.4.2 h8cc25b3_0 zipfile-deflate64 0.2.0 py39ha55989b_4 conda-forge zipp 3.17.0 pyhd8ed1ab_0 conda-forge zstd 1.5.5 h12be248_0 conda-forge ```
sannawi commented 10 months ago

Hi!

I am experiencing the same issue, but with importing Ecoinvent 3.7.1. I managed to import v3.9.1 but when trying to import v3.7.1, I get a window saying that "The import failed as required biosphere exchanges are missing from the biosphere3 database. Please update the biosphere by using 'File' -> 'Update biosphere...'". I tried to update the biosphere but the issue remains.

Thank you in advance! Best, Sanna Wickerts

marc-vdm commented 10 months ago

Hi Sanna,

While we work on a proper solution, I suggest this temporary solution:

  1. Create a new environment with conda create -n ab28 -c conda-forge activity-browser=2.8.0 bw2io=0.8.8 (don't worry, all your databases and projects are safe)
  2. In the new ab28 environment, create a new project, import the biosphere and import the database there
  3. You can now use your normal environment again to work with your 3.7.1 project/database
sannawi commented 10 months ago

Hi Marc,

Ok great, thank you! I will try that.

Best, Sanna

From: Marc van der Meide @.> Date: Wednesday, 29 November 2023 at 16:35 To: LCA-ActivityBrowser/activity-browser @.> Cc: Sanna Wickerts @.>, Comment @.> Subject: Re: [LCA-ActivityBrowser/activity-browser] ecoinvent importer failing with bw2io=0.8.10 (Issue #1137)

Hi Sanna,

While we work on a proper solution, I suggest this temporary solution:

  1. Create a new environment with conda create -n ab28 -c conda-forge activity-browser=2.8.0 bw2io=0.8.8 (don't worry, all your databases and projects are safe)
  2. In the new ab28 environment, create a new project, import the biosphere and import the database there
  3. You can now use your normal environment again to work with your 3.7.1 project/database

— Reply to this email directly, view it on GitHubhttps://github.com/LCA-ActivityBrowser/activity-browser/issues/1137#issuecomment-1832147901, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AVTG5MUPUEWXBAH4WHHMXUDYG5I27AVCNFSM6AAAAAA7X6VZG6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZSGE2DOOJQGE. You are receiving this because you commented.Message ID: @.***>

sannawi commented 10 months ago

Hi again,

Unfortunately, I don't think it worked? I am not sure as I am very new to coding and Brightway.

Here is the code:

Last login: Thu Nov 30 08:21:36 on console

The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
(base) CM-C02Z24VLL412:~ sannajo$ conda create -n ab28 -c conda-forge activity-browser=2.8.0 bw2io=0.8.8
Error while loading conda entry point: conda-libmamba-solver (dlopen(/Users/sannajo/anaconda3/lib/python3.11/site-packages/libmambapy/bindings.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libarchive.19.dylib
  Referenced from: <4F399D22-A23C-3D44-9152-8ED3D55C3724> /Users/sannajo/anaconda3/lib/libmamba.2.0.0.dylib
  Reason: tried: '/Users/sannajo/anaconda3/lib/libarchive.19.dylib' (no such file), '/Users/sannajo/anaconda3/lib/python3.11/site-packages/libmambapy/../../../libarchive.19.dylib' (no such file), '/Users/sannajo/anaconda3/lib/python3.11/site-packages/libmambapy/../../../libarchive.19.dylib' (no such file), '/Users/sannajo/anaconda3/bin/../lib/libarchive.19.dylib' (no such file), '/Users/sannajo/anaconda3/bin/../lib/libarchive.19.dylib' (no such file), '/usr/local/lib/libarchive.19.dylib' (no such file), '/usr/lib/libarchive.19.dylib' (no such file, not in dyld cache))

CondaValueError: You have chosen a non-default solver backend (libmamba) but it was not recognized. Choose one of: classic

(base) CM-C02Z24VLL412:~ sannajo$ 
marc-vdm commented 10 months ago

@haasad Could you look at the solver issue above when you have time?

haasad commented 10 months ago

Most likely an old conda version. You can update conda with this command:

conda update -n base conda

After that the install command should run.

@marc-vdm IIRC there used to be a section about keeping the conda version up to date in the README, looks like that got lost somewhere along the way

marc-vdm commented 10 months ago

Hi @sannawi

Your issue will be resolved in the next version of Activity Browser (2.9.3), which we will release this or next week.

Keep an eye out on the updates mailing list or the releases page to see when it becomes available.