search

euronion / trace

15 stars 3 forks source link

environment.yml ResolvePackageNotFound #7

Closed MurkyTheMurloc closed 2 years ago

MurkyTheMurloc commented 2 years ago

Error on windows, on ubuntu endless loop of examining conflicts between packages

$ conda env create -f envs/environment.yaml Collecting package metadata (repodata.json): ...working... done Solving environment: ...working... failed

ResolvePackageNotFound:

What we have tried so far:

We are looking forward to finding a solution!

euronion commented 2 years ago

Hi there,

I haven't used this branch in a while now and am not sure what the issue is caused by. The environment.yaml is definetly overloaded and has a lot of aux packages and dependencies in it which it shouldn't. This in combination with the build numbers probably breaks the environment solving.

You could try this one (I haven't tested it with the full workflow) but it is what conda env export --from-history --no-build yields:

name: trace
channels:
  - https://conda.anaconda.org/gurobi
  - conda-forge
  - defaults
dependencies:
  - python==3.7
  - pypsa
  - coincbc
  - matplotlib
  - seaborn
  - xarray
  - plotly_express
  - snakemake==5.26.1
  - notebook
  - h5py
  - gurobi
  - snakefmt
  - ca-certificates
  - certifi
  - openssl
  - pre-commit
  - hvplot
  - selenium
  - firefox
  - geckodriver
  - pycountry

Alternatively if you want to use the workflow and model you can also have a look at the following branch on which I am currently working:

https://github.com/euronion/trace/tree/replace-gegis

Installation there should work (the environment.yaml is as lean as it is supposed to be). That branch of the model no longer uses GEGIS and uses atlite instead, thus is Python only (no need for Julia or Julia complications). I'm already using that branch productively and was planning on merging it in the next months.

HTH

MurkyTheMurloc commented 2 years ago

Well the Problem more likely is that there are some spelling miastages in the yaml. Like openssll for example ore jpegd. Could you double check this please?

euronion commented 2 years ago

Try this one then:

name: trace
channels:
  - bioconda
  - plotly
  - gurobi
  - conda-forge
  - https://conda.anaconda.org/gurobi
  - defaults
dependencies:
  - _libgcc_mutex=0.1
  - _openmp_mutex=4.5
  - aioeasywebdav=2.4.0
  - aiohttp=3.7.2
  - amply=0.1.4
  - appdirs=1.4.4
  - argon2-cffi=20.1.0
  - async-timeout=3.0.1
  - async_generator=1.10
  - attmap=0.13.0
  - attrs=20.3.0
  - backcall=0.2.0
  - backports=1.0
  - backports.functools_lru_cache=1.6.1
  - bcrypt=3.2.0
  - black=20.8b1
  - bleach=3.2.1
  - blosc=1.20.1
  - bokeh=2.3.3
  - boto=2.49.0
  - boto3=1.16.19
  - botocore=1.19.19
  - brotlipy=0.7.0
  - bz2file=0.98
  - bzip2=1.0.8
  - c-ares=1.16.1
  - ca-certificates=2021.10.26
  - cached-property=1.5.2
  - cachetools=4.1.1
  - cairo=1.16.0
  - cartopy=0.18.0
  - certifi=2021.10.8
  - cffi=1.14.3
  - cfgv=3.3.1
  - cftime=1.3.0
  - chardet=3.0.4
  - click=7.1.2
  - coincbc=2.10.5
  - colorama=0.4.4
  - colorcet=3.0.0
  - configargparse=1.2.3
  - cryptography=3.2.1
  - curl=7.71.1
  - cycler=0.10.0
  - dataclasses=0.8
  - datrie=0.8.2
  - dbus=1.13.18
  - decorator=4.4.2
  - defusedxml=0.6.0
  - distlib=0.3.3
  - docutils=0.16
  - dropbox=10.9.0
  - editdistance=0.5.3
  - entrypoints=0.3
  - expat=2.2.10
  - fftw=3.3.8
  - filechunkio=1.8
  - filelock=3.0.12
  - firefox=94.0.2
  - fontconfig=2.13.1
  - freetype=2.10.4
  - ftputil=4.0.0
  - geckodriver=0.30.0
  - geos=3.8.1
  - gettext=0.19.8.1
  - ghostscript=9.53.3
  - giflib=5.1.9
  - gitdb=4.0.5
  - gitpython=3.1.11
  - glib=2.66.1
  - google-api-core=1.23.0
  - google-api-python-client=1.12.5
  - google-auth=1.23.0
  - google-auth-httplib2=0.0.4
  - google-cloud-core=1.4.3
  - google-cloud-storage=1.32.0
  - google-crc32c=1.0.0
  - google-resumable-media=1.1.0
  - googleapis-common-protos=1.52.0
  - graphite2=1.3.14
  - graphviz=2.40.1
  - grpcio=1.33.2
  - gst-plugins-base=1.14.0
  - gstreamer=1.14.0
  - gurobi=9.5.0
  - h5py=3.1.0
  - harfbuzz=2.4.0
  - hdf4=4.2.13
  - hdf5=1.10.6
  - holoviews=1.14.6
  - httplib2=0.18.1
  - hvplot=0.7.3
  - icu=58.2
  - identify=2.1.4
  - idna=2.10
  - imagemagick=7.0.8_11
  - importlib-metadata=1.7.0
  - importlib_metadata=1.7.0
  - iniconfig=1.1.1
  - ipykernel=5.3.4
  - ipython=7.19.0
  - ipython_genutils=0.2.0
  - jbig=2.1
  - jedi=0.17.2
  - jinja2=2.11.2
  - jmespath=0.10.0
  - jpeg=9d
  - jsonschema=3.2.0
  - jupyter_client=6.1.7
  - jupyter_core=4.6.3
  - jupyterlab_pygments=0.1.2
  - kiwisolver=1.3.1
  - krb5=1.17.1
  - lcms2=2.11
  - ld_impl_linux-64=2.33.1
  - libblas=3.9.0
  - libcblas=3.9.0
  - libcrc32c=1.1.1
  - libcurl=7.71.1
  - libedit=3.1.20191231
  - libev=4.33
  - libffi=3.3
  - libgcc-ng=11.2.0
  - libgfortran-ng=7.5.0
  - libgfortran4=7.5.0
  - libgomp=11.2.0
  - liblapack=3.9.0
  - libnetcdf=4.7.4
  - libnghttp2=1.41.0
  - libopenblas=0.3.12
  - libpng=1.6.37
  - libprotobuf=3.13.0.1
  - libsodium=1.0.18
  - libssh2=1.9.0
  - libstdcxx-ng=11.2.0
  - libtiff=4.1.0
  - libtool=2.4.6
  - libuuid=2.32.1
  - libwebp=0.5.2
  - libwebp-base=1.1.0
  - libxcb=1.14
  - libxml2=2.9.10
  - logmuse=0.2.6
  - lz4-c=1.9.2
  - lzo=2.10
  - markdown=3.3.4
  - markupsafe=1.1.1
  - matplotlib=3.3.3
  - matplotlib-base=3.3.3
  - mistune=0.8.4
  - mock=4.0.2
  - more-itertools=8.7.0
  - multidict=5.0.0
  - mypy_extensions=0.4.3
  - nbclient=0.5.1
  - nbconvert=6.0.7
  - nbformat=5.0.8
  - ncurses=6.2
  - nest-asyncio=1.4.3
  - netcdf4=1.5.4
  - networkx=2.5
  - nodeenv=1.6.0
  - nose=1.3.7
  - notebook=6.1.5
  - numexpr=2.7.1
  - numpy=1.19.4
  - oauth2client=4.1.3
  - olefile=0.46
  - openjpeg=2.3.1
  - openssl=1.1.1l
  - packaging=20.4
  - pandas=1.1.4
  - pandoc=2.11.1.1
  - pandocfilters=1.4.3
  - panel=0.12.1
  - pango=1.40.14
  - param=1.12.0
  - paramiko=2.7.2
  - parso=0.7.1
  - pathspec=0.8.1
  - patsy=0.5.1
  - pcre=8.44
  - peppy=0.31.0
  - perl=5.26.2
  - pexpect=4.8.0
  - pickleshare=0.7.5
  - pillow=8.0.1
  - pip=20.2.4
  - pixman=0.38.0
  - pkg-config=0.29.2
  - plotly=4.12.0
  - plotly_express=0.4.1
  - pluggy=0.13.1
  - ply=3.11
  - pre-commit=2.15.0
  - prettytable=2.0.0
  - proj=7.1.1
  - prometheus_client=0.9.0
  - prompt-toolkit=3.0.8
  - protobuf=3.13.0.1
  - psutil=5.7.3
  - ptyprocess=0.6.0
  - pulp=2.3.1
  - py=1.10.0
  - pyasn1=0.4.8
  - pyasn1-modules=0.2.8
  - pycountry=20.7.3
  - pycparser=2.20
  - pyct=0.4.6
  - pyct-core=0.4.6
  - pygments=2.7.2
  - pygraphviz=1.6
  - pynacl=1.4.0
  - pyomo=5.7.1
  - pyopenssl=19.1.0
  - pyparsing=2.4.7
  - pypsa=0.17.1
  - pyqt=5.9.2
  - pyrsistent=0.17.3
  - pysftp=0.2.9
  - pyshp=2.1.2
  - pysocks=1.7.1
  - pytables=3.6.1
  - pytest=6.2.2
  - python=3.7.9
  - python-dateutil=2.8.1
  - python-irodsclient=0.8.2
  - python_abi=3.7
  - pytz=2020.4
  - pyutilib=6.0.0
  - pyviz_comms=2.1.0
  - pyyaml=5.3.1
  - pyzmq=20.0.0
  - qt=5.9.7
  - ratelimiter=1.2.0
  - readline=8.0
  - regex=2021.3.17
  - requests=2.25.0
  - retrying=1.3.3
  - rsa=4.6
  - s3transfer=0.3.3
  - scipy=1.5.3
  - seaborn=0.11.0
  - seaborn-base=0.11.0
  - selenium=3.141.0
  - send2trash=1.5.0
  - setuptools=50.3.1
  - shapely=1.7.1
  - simplejson=3.17.2
  - sip=4.19.8
  - six=1.15.0
  - slacker=0.14.0
  - smart_open=4.2.0
  - smmap=3.0.4
  - snakefmt=0.4.0
  - snakemake=6.2.1
  - snakemake-minimal=6.2.1
  - sqlite=3.33.0
  - statsmodels=0.12.1
  - stopit=1.1.2
  - terminado=0.9.1
  - testpath=0.4.4
  - tk=8.6.10
  - toml=0.10.2
  - toposort=1.5
  - tornado=6.1
  - tqdm=4.62.3
  - traitlets=5.0.5
  - typed-ast=1.4.2
  - typing-extensions=3.7.4.3
  - typing_extensions=3.7.4.3
  - ubiquerg=0.6.1
  - uritemplate=3.0.1
  - urllib3=1.25.11
  - veracitools=0.1.3
  - virtualenv=20.4.7
  - wcwidth=0.2.5
  - webencodings=0.5.1
  - wheel=0.35.1
  - wrapt=1.12.1
  - xarray=0.16.1
  - xmlrunner=1.7.7
  - xorg-kbproto=1.0.7
  - xorg-libice=1.0.10
  - xorg-libsm=1.2.3
  - xorg-libx11=1.6.12
  - xorg-libxau=1.0.9
  - xorg-libxext=1.3.4
  - xorg-libxpm=3.5.13
  - xorg-libxrender=0.9.10
  - xorg-libxt=1.1.5
  - xorg-renderproto=0.11.1
  - xorg-xextproto=7.3.0
  - xorg-xproto=7.0.31
  - xz=5.2.5
  - yaml=0.2.5
  - yarl=1.5.1
  - zeromq=4.3.3
  - zipp=3.4.0
  - zlib=1.2.11
  - zstd=1.4.5

Not sure where those spelling mistakes come from, maybe from sed-ing the package versions away manually...

Let me know if that works.

MurkyTheMurloc commented 2 years ago

Hi, thanks for the advice. Unfortunately, it didn't work. Screenshot from 2022-06-26 21-07-22

euronion commented 2 years ago

Hm. Did you also try the one above without frozen package versions and dependency packages explicitly stated?

Cf. https://github.com/euronion/trace/issues/7#issuecomment-1159723056

MurkyTheMurloc commented 2 years ago

Hi, yes, I've also tried that with conda and mamba: image

euronion commented 2 years ago

Ok, that's a different error and one we can easily adress. The bioconda channel is missing above. Try again with this one:

name: trace
channels:
  - bioconda
  - https://conda.anaconda.org/gurobi
  - conda-forge
  - defaults
dependencies:
  - python
  - pypsa
  - coincbc
  - matplotlib
  - seaborn
  - xarray
  - plotly_express
  - snakemake
  - notebook
  - h5py
  - gurobi
  - snakefmt
  - ca-certificates
  - certifi
  - openssl
  - pre-commit
  - hvplot
  - selenium
  - firefox
  - geckodriver
  - pycountry

No guarantee this will work, but it should get us one step further and get rid of the error you saw.

MurkyTheMurloc commented 2 years ago

Thanks a lot, creating the environment worked now!

euronion commented 2 years ago

Great! I hope the environment works without issues.

MurkyTheMurloc commented 2 years ago

Thanks again for helping me set up the environment.

Could you point me in a direction on how to use it to create an ESC? I’m fairly new at this but try to use it for my thesis. I’ve followed the steps in “trace” and created input data with GlobalEnergyGIS/Synde respositories before. What steps am I missing before solving scenarios via snakemake?