search

mamba-org / micromamba-docker

Rapid builds of small Conda-based containers using micromamba.
https://micromamba-docker.readthedocs.io/
Apache License 2.0
258 stars 45 forks source link

BUG: Read environment 'dependencies' as list of strings from environment.yml #11

Closed rdmolony closed 3 years ago

rdmolony commented 3 years ago

I'm currently getting ERROR Could not read environment 'dependencies' as list of strings from /tmp/environment.yml with the following Dockerfile

FROM mambaorg/micromamba:0.8.2

# create the conda environment and install cea...
# && remove Python Bytecode files & Static Libraries to save 400MB in space
COPY environment.yml /tmp/environment.yml
RUN micromamba install -y -n cea -f /tmp/environment.yml \
    && rm /opt/conda/pkgs/cache/* \
    && find /opt/conda/ -follow -type f -name '*.a' -delete \
    && find /opt/conda/ -follow -type f -name '*.pyc' -delete

& environment.yml

name: cea
channels:
  - conda-forge
  - defaults
  - dlr-sc
  - pythonocc
  - oce
dependencies:
  - affine=2.3.0
  - attrs=19.3.0
  - backcall=0.2.0
  - beautifulsoup4=4.9.1
  - blas=1.0
  - bleach=3.1.5
  - blosc=1.20.0
  - boost-cpp=1.72.0
  - branca=0.4.1
  - brotlipy=0.7.0
  - bs4=4.9.1
  - bzip2=1.0.8
  - ca-certificates=2020.6.20
  - certifi=2020.6.20
  - cffi=1.14.1
  - cfitsio=3.470
  - chardet=3.0.4
  - click=7.1.2
  - click-plugins=1.1.1
  - cligj=0.5.0
  - colorama=0.4.3
  - cryptography=3.0
  - curl=7.71.1
  - cvxopt=1.2.5
  - cycler=0.10.0
  - decorator=4.4.2
  - defusedxml=0.6.0
  - deprecated=1.2.10
  - descartes=1.1.0
  - double-conversion=3.1.5
  - dsdp=5.8
  - eigen=3.3.7
  - entrypoints=0.3
  - esda=2.3.1
  - expat=2.2.9
  - fastcache=1.1.0
  - ffmpeg=4.3
  - fftw=3.3.8
  - fiona=1.8.9.post2
  - folium=0.11.0
  - fontconfig=2.13.1
  - freeimage=3.18.0
  - freetype=2.10.2
  - freexl=1.0.5
  - future=0.18.2
  - gdal=2.4.4
  - geopandas=0.8.1
  - geos=3.8.1
  - geotiff=1.5.1
  - gettext=0.19.8.1
  - gevent=20.6.2
  - gevent-websocket=0.10.1
  - giddy=2.3.3
  - gl2ps=1.4.2
  - glew=2.1.0
  - glib=2.65.0
  - glpk=4.65
  - greenlet=0.4.16
  - gsl=2.6
  - hdf4=4.2.13
  - hdf5=1.10.5
  - icc_rt=2019.0.0
  - icu=64.2
  - idna=2.10
  - importlib-metadata=1.7.0
  - importlib_metadata=1.7.0
  - inequality=1.0.0
  - intel-openmp=2020.1
  - ipdb=0.13.0
  - ipykernel=5.3.4
  - ipython=7.16.1
  - ipython_genutils=0.2.0
  - ipywidgets=7.5.1
  - jedi=0.17.2
  - jinja2=2.11.2
  - joblib=0.16.0
  - jpeg=9d
  - jsoncpp=1.8.4
  - jsonschema=3.2.0
  - jupyter
  - kealib=1.4.13
  - kiwisolver=1.2.0
  - krb5=1.17.1
  - libblas=3.8.0
  - libcblas=3.8.0
  - libclang=9.0.1
  - libcurl=7.71.1
  - libffi=3.2.1
  - libgdal=2.4.4
  - libiconv=1.15
  - libkml=1.3.0
  - liblapack=3.8.0
  - libllvm9=9.0.1
  - libnetcdf=4.7.4
  - libogg=1.3.2
  - libpng=1.6.37
  - libpq=12.3
  - libpysal=4.3.0
  - libsodium=1.0.17
  - libspatialindex=1.9.3
  - libspatialite=4.3.0a
  - libssh2=1.9.0
  - libtheora=1.1.1
  - libtiff=4.1.0
  - libwebp-base=1.1.0
  - libxml2=2.9.10
  - llvmlite=0.33.0
  - loguru=0.5.0
  - lz4-c=1.9.2
  - m2w64-expat=2.1.1
  - m2w64-gcc-libgfortran=5.3.0
  - m2w64-gcc-libs=5.3.0
  - m2w64-gcc-libs-core=5.3.0
  - m2w64-gettext=0.19.7
  - m2w64-gmp=6.1.0
  - m2w64-libiconv=1.14
  - m2w64-libwinpthread-git=5.0.0.4634.697f757
  - m2w64-xz=5.2.2
  - mapclassify=2.3.0
  - markupsafe=1.1.1
  - matplotlib=3.3.0
  - matplotlib-base=3.3.0
  - mgwr=2.1.1
  - mistune=0.8.4
  - mkl=2020.1
  - mkl-service=2.3.0
  - mock=4.0.2
  - mpmath=1.1.0
  - msys2-conda-epoch=20160418
  - munch=2.5.0
  - nbconvert=5.6.1
  - nbformat=5.0.7
  - networkx=2.4
  - nose=1.3.7
  - notebook=6.0.3
  - numba=0.50.1
  - numexpr=2.7.1
  - numpy=1.19.1
  - occt=7.4.0
  - olefile=0.46
  - openjpeg=2.3.1
  - openssl=1.1.1g
  - osmnx=0.15.1
  - packaging=20.4
  - palettable=3.3.0
  - pandas=1.0.5
  - pandoc=2.10.1
  - pandocfilters=1.4.2
  - parso=0.7.1
  - patsy=0.5.1
  - pcre=8.44
  - pickleshare=0.7.5
  - pillow=7.2.0
  - pip=20.1.1
  - pointpats=2.1.0
  - poppler=0.67.0
  - poppler-data=0.4.9
  - postgresql=12.3
  - proj=7.0.0
  - prometheus_client=0.8.0
  - prompt-toolkit=3.0.5
  - pugixml=1.10
  - pycparser=2.20
  - pygments=2.6.1
  - pyopenssl=19.1.0
  - pyparsing=2.4.7
  - pyproj=2.6.1.post1
  - pyqt=5.12.3
  - pyrsistent=0.16.0
  - pysal=2.1.0
  - pysocks=1.7.1
  - pytables=3.6.1
  - python=3.7.8
  - python-dateutil=2.8.1
  - python_abi=3.7
  - pythonocc-core=7.4.0
  - pytz=2020.1
  - pywin32=227
  - pywinpty=0.5.7
  - pyzmq=19.0.1
  - qt=5.12.5
  - quantecon=0.4.8
  - rasterio=1.1.5
  - rasterstats=0.14.0
  - requests=2.24.0
  - rtree=0.9.4
  - scikit-learn=0.23.1
  - scipy=1.5.0
  - seaborn=0.10.1
  - seaborn-base=0.10.1
  - segregation=1.3.0
  - send2trash=1.5.0
  - setuptools=49.2.0
  - shapely=1.7.0
  - simplejson=3.17.2
  - six=1.15.0
  - snuggs=1.4.7
  - soupsieve=2.0.1
  - spaghetti=1.5.0
  - spglm=1.0.7
  - spint=1.0.6
  - splot=1.1.3
  - spreg=1.1.1
  - spvcm=0.3.0
  - sqlite=3.32.3
  - statsmodels=0.11.1
  - sympy=1.6.1
  - tbb=2020.1
  - terminado=0.8.3
  - testpath=0.4.4
  - threadpoolctl=2.1.0
  - tiledb=2.0.6
  - tk=8.6.10
  - tobler=0.3.1
  - tornado=6.0.4
  - tqdm=4.48.0
  - traitlets=4.3.3
  - urllib3=1.25.10
  - utfcpp=3.1.1
  - vc=14.1
  - vs2015_runtime=14.16.27012
  - vtk=8.2.0
  - wcwidth=0.2.5
  - webencodings=0.5.1
  - wheel=0.34.2
  - widgetsnbextension=3.5.1
  - win32_setctime=1.0.1
  - win_inet_pton=1.1.0
  - wincertstore=0.2
  - winpty=0.4.3
  - wntr=0.2.2.1
  - wrapt=1.12.1
  - xerces-c=3.2.2
  - xz=5.2.5
  - zeromq=4.3.2
  - zipp=3.1.0
  - zlib=1.2.11
  - zope.event=4.4
  - zope.interface=5.1.0
  - zstd=1.4.5
  - pip:
    - alabaster==0.7.12
    - aniso8601==8.0.0
    - arviz==0.9.0
    - babel==2.8.0
    # - cea-workflow-utils==1.0.0
    - cftime==1.2.1
    - cityenergyanalyst==3.8.0a0
    - colorlover==0.3.0
    - cufflinks==0.17.3
    - deap==1.3.1
    - docutils==0.16
    - ephem==3.7.7.1
    - et-xmlfile==1.0.1
    - fastprogress==0.2.3
    - flask==1.1.2
    - flask-restplus==0.13.0
    - flask-socketio==4.3.1
    - h5py==2.10.0
    - imagesize==1.2.0
    - itsdangerous==1.1.0
    - jdcal==1.4.1
    - keyring==21.2.1
    - lxml==4.5.2
    - netcdf4==1.5.4
    - openpyxl==3.0.4
    - pkginfo==1.5.0.1
    - plotly==4.9.0
    - psutil==5.7.2
    - py4design-cea==0.30
    - pycollada==0.7.1
    - pymc3==3.9.2
    - pyqt5-sip==4.19.18
    - pyqtwebengine==5.12.1
    - pyshp==2.1.0
    - python-engineio==3.13.1
    - python-socketio==4.6.0
    - pywin32-ctypes==0.2.0
    - pyyaml==5.3.1
    - readme-renderer==26.0
    - requests-toolbelt==0.9.1
    - retrying==1.3.3
    - rfc3986==1.4.0
    - salib==1.3.11
    - snowballstemmer==2.0.0
    - staticmap==0.5.4
    - theano==1.0.4
    - timezonefinder==4.4.0
    - twine==3.2.0
    - typing-extensions==3.7.4.2
    - utm==0.5.0
    - werkzeug==1.0.1
    - xarray==0.16.0
    - xlrd==1.2.0
    - xlwt==1.3.0
prefix: C:\Users\darthoma\Miniconda2\envs\cea-3.8.0a1

Full error message:

 => [internal] load build definition from Dockerfile.micromamba                                             0.0s
 => => transferring dockerfile: 48B                                                                         0.0s
 => [internal] load .dockerignore                                                                           0.0s
 => => transferring context: 2B                                                                             0.0s
 => [internal] load metadata for docker.io/mambaorg/micromamba:0.8.2                                        1.7s
 => [auth] mambaorg/micromamba:pull token for registry-1.docker.io                                          0.0s
 => [internal] load build context                                                                           0.1s
 => => transferring context: 44.54kB                                                                        0.1s
 => [ 1/11] FROM docker.io/mambaorg/micromamba:0.8.2@sha256:5eae5f08305d921145942fd31b12f4060fd4fc7ef5cd35  0.0s
 => [ 5/11] COPY ./docker/environment.yml /tmp/environment.yml 
 => ERROR [ 6/11] RUN micromamba install -y -n cea -f /tmp/environment.yml     && rm /opt/conda/pkgs/cache  0.4s
------                                                                                                           
 > [ 6/11] RUN micromamba install -y -n cea -f /tmp/environment.yml     && rm /opt/conda/pkgs/cache/*     && find /opt/conda/ -follow -type f -name '*.a' -delete     && find /opt/conda/ -follow -type f -name '*.pyc' -delete:
#11 0.355 ERROR   Could not read environment 'dependencies' as list of strings from /tmp/environment.yml
------
executor failed running [/bin/bash -c micromamba install -y -n cea -f /tmp/environment.yml     && rm /opt/conda/pkgs/cache/*     && find /opt/conda/ -follow -type f -name '*.a' -delete     && find /opt/conda/ -follow -type f -name '*.pyc' -delete]: exit code: 1

Also tried:

name: cea
channels:
- conda-forge
- defaults
dependencies:
- cvxopt
- deap
- docutils
- ephem
- flask
- flask-restplus
- flask-socketio
- gevent
- h5py
- ipykernel
- ipython
- jupyter
- lxml
- openpyxl
- plotly
- salib
- sphinx
- sympy
- theano
- utm
- wntr
- xlrd
- xlwt
- descartes
- fiona
- folium
- gdal
- geopandas
- geopy
- geos
- hdf4
- hdf5
- jinja2
- json-c
- llvmlite
- matplotlib
- networkx
- numba
- numpy
- osmnet
- osmnx
- pandana
- pandas
- pcre
- pip
- proj
- psutil
- pyproj
- pysal<2.3.0
- python>=3.7
- pythonocc-core
- pytz
- pyyaml
- scikit-learn
- scipy
- seaborn
- setuptools
- shapely
- urbanaccess
- yaml
- timezonefinder
- twine
- pymc3
- python-cufflinks
- gevent-websocket
- pip:
  - opencv-contrib-python
  - staticmap
  - py4design-cea

Please let me know if you see anything obvious as am very interested in using micromamba to slim down this docker image instead of a multistage build! Thanks for creating / maintaining it, seems like a really nice alternative to miniconda here!

wholtz commented 3 years ago

Hi @rdmolony. Thanks for the interest in micromamba. I'm not sure why you are getting that specific error, but here are a few issues I see:

  1. Currently, pip isn't supported by micromamba - see mamba issue #589.
  2. Use the 'base' environment if you are only going to have a single conda environment within your container. If you use another environment name you'll need to run micromamba create ... and micromamba activate ....
rdmolony commented 3 years ago

Thanks a lot @wholtz, that clears it up & sorry I should have spotted that in the mamba repo!