search

conda-forge / geopandas-feedstock

A conda-smithy repository for geopandas.
BSD 3-Clause "New" or "Revised" License
16 stars 14 forks source link

new PROJ_LIB handling breaks Pyproj, Fiona #63

Closed hyperknot closed 5 years ago

hyperknot commented 5 years ago

Issue:

I've just recreated a Docker image which had the following lines to install geopandas in a clean Ubuntu 16.04 system. This has been working perfectly for a year before yesterday.

wget --quiet https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh && \
    /bin/bash ~/miniconda.sh -b -p /opt/conda && \
    conda update conda -y && \
    conda config --add channels conda-forge && \
    conda config --set channel_priority strict && \
    conda install \
        geopandas numexpr bottleneck

However, as of yesterday, everything is broken.

First, I get lots of warning lines of the one below:

proj_create: Open of /opt/conda/share/proj failed
proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode

I've opened this issue in https://github.com/conda-forge/pyproj-feedstock/issues/47 however I've just been directed over to Fiona, so I've opened one there as well: https://github.com/conda-forge/fiona-feedstock/issues/137

However the big problem is actually not these warning lines, but that geopandas / pyproj actually seg faults. I've opened this issue for it: https://github.com/pyproj4/pyproj/issues/415

Now, I was able to downgrade pyproj to 2.2.2 using conda install 'pyproj < 2.3'. This gave me a better version of the segmentation fault, which is a nice Python exception with a warning message at the bottom (detailed in the pyproj issue).

"Valid PROJ data directory not found. "
pyproj.exceptions.DataDirError: Valid PROJ data directory not found. Either set the path using the environmental variable PROJ_LIB or with `pyproj.datadir.set_data_dir`.

So I believe the core is with the geopandas conda install, and the intertwined Python modules. To say it shortly, it's broken as it is now, on Ubuntu at least.

Replicating is very simple, you can simply run the afore mentioned command in a Docker image and it'll replicate it 100%. The seg fault I cannot replicate, as it only happens with some input files.

I'm looking for ways to make this work, but I don't even know where to start. What packages should I downgrade?


Environment (conda list):

``` # packages in environment at /opt/conda: # # Name Version Build Channel _libgcc_mutex 0.1 main asn1crypto 0.24.0 py37_1003 conda-forge attrs 19.1.0 py_0 conda-forge backcall 0.1.0 pypi_0 pypi boost-cpp 1.70.0 ha2d47e9_1 conda-forge bottleneck 1.2.1 py37h3010b51_1001 conda-forge bzip2 1.0.8 h516909a_0 conda-forge ca-certificates 2019.6.16 hecc5488_0 conda-forge cachetools 3.1.1 pypi_0 pypi cairo 1.16.0 h18b612c_1001 conda-forge cairocffi 1.0.2 pypi_0 pypi cairosvg 2.4.1 pypi_0 pypi certifi 2019.6.16 py37_1 conda-forge cffi 1.12.3 py37h8022711_0 conda-forge cfitsio 3.470 hb60a0a2_2 conda-forge chardet 3.0.4 py37_1003 conda-forge click 7.0 py_0 conda-forge click-plugins 1.1.1 py_0 conda-forge cligj 0.5.0 py_0 conda-forge conda 4.7.11 py37_0 conda-forge conda-package-handling 1.4.1 py37_0 conda-forge cryptography 2.7 py37h72c5cf5_0 conda-forge cssselect2 0.2.1 pypi_0 pypi curl 7.65.3 hf8cf82a_0 conda-forge decorator 4.4.0 pypi_0 pypi defusedxml 0.6.0 pypi_0 pypi expat 2.2.5 he1b5a44_1003 conda-forge fiona 1.8.6 py37h5f563d9_4 conda-forge fontconfig 2.13.1 he4413a7_1000 conda-forge freetype 2.10.0 he983fc9_1 conda-forge freexl 1.0.5 h14c3975_1002 conda-forge gdal 2.4.2 py37h0b7157c_7 conda-forge geoalchemy2 0.6.3 pypi_0 pypi geographiclib 1.49 pypi_0 pypi geojson 2.5.0 pypi_0 pypi geopandas 0.5.1 py_0 conda-forge geopy 1.20.0 pypi_0 pypi geos 3.7.2 he1b5a44_1 conda-forge geotiff 1.5.1 hcd53e25_3 conda-forge gettext 0.19.8.1 hc5be6a0_1002 conda-forge giflib 5.1.7 h516909a_1 conda-forge glib 2.58.3 h6f030ca_1002 conda-forge google-api-core 1.14.2 pypi_0 pypi google-auth 1.6.3 pypi_0 pypi google-cloud-core 1.0.3 pypi_0 pypi google-cloud-firestore 1.4.0 pypi_0 pypi google-cloud-storage 1.18.0 pypi_0 pypi google-resumable-media 0.3.3 pypi_0 pypi googleapis-common-protos 1.6.0 pypi_0 pypi grpcio 1.23.0 pypi_0 pypi hdf4 4.2.13 h9a582f1_1002 conda-forge hdf5 1.10.4 nompi_h3c11f04_1106 conda-forge html5lib 1.0.1 pypi_0 pypi icu 58.2 hf484d3e_1000 conda-forge idna 2.8 py37_1000 conda-forge ipython 7.7.0 pypi_0 pypi ipython-genutils 0.2.0 pypi_0 pypi jedi 0.15.1 pypi_0 pypi jpeg 9c h14c3975_1001 conda-forge json-c 0.13.1 h14c3975_1001 conda-forge kealib 1.4.10 h1978553_1003 conda-forge krb5 1.16.3 h05b26f9_1001 conda-forge libarchive 3.3.3 hb44662c_1005 conda-forge libblas 3.8.0 12_openblas conda-forge libcblas 3.8.0 12_openblas conda-forge libcurl 7.65.3 hda55be3_0 conda-forge libdap4 3.20.2 hd48c02d_1000 conda-forge libedit 3.1.20170329 hf8c457e_1001 conda-forge libffi 3.2.1 he1b5a44_1006 conda-forge libgcc-ng 9.1.0 hdf63c60_0 libgdal 2.4.2 h182c168_7 conda-forge libgfortran-ng 7.3.0 hdf63c60_0 libiconv 1.15 h516909a_1005 conda-forge libkml 1.3.0 h4fcabce_1010 conda-forge liblapack 3.8.0 12_openblas conda-forge libnetcdf 4.6.2 hbdf4f91_1001 conda-forge libopenblas 0.3.7 h6e990d7_1 conda-forge libpng 1.6.37 hed695b0_0 conda-forge libpq 11.5 hd9ab2ff_1 conda-forge libspatialindex 1.9.0 he1b5a44_1 conda-forge libspatialite 4.3.0a h57ae47a_1030 conda-forge libssh2 1.8.2 h22169c7_2 conda-forge libstdcxx-ng 9.1.0 hdf63c60_0 libtiff 4.0.10 h57b8799_1003 conda-forge libuuid 2.32.1 h14c3975_1000 conda-forge libxcb 1.13 h14c3975_1002 conda-forge libxml2 2.9.9 h13577e0_2 conda-forge lz4-c 1.8.3 he1b5a44_1001 conda-forge lzo 2.10 h14c3975_1000 conda-forge mako 1.1.0 pypi_0 pypi markupsafe 1.1.1 pypi_0 pypi munch 2.3.2 py_0 conda-forge ncurses 6.1 hf484d3e_1002 conda-forge numexpr 2.7.0 py37hb3f55d8_0 conda-forge numpy 1.17.1 py37h95a1406_0 conda-forge openjpeg 2.3.1 h58a6597_0 conda-forge openssl 1.1.1c h516909a_0 conda-forge pandas 0.25.1 py37hb3f55d8_0 conda-forge parso 0.5.1 pypi_0 pypi pcre 8.41 hf484d3e_1003 conda-forge pendulum 2.0.5 pypi_0 pypi pexpect 4.7.0 pypi_0 pypi pickleshare 0.7.5 pypi_0 pypi pillow 6.1.0 pypi_0 pypi pip 19.2.3 py37_0 conda-forge pixman 0.38.0 h516909a_1003 conda-forge poppler 0.67.0 ha967d66_7 conda-forge poppler-data 0.4.9 1 conda-forge postgresql 11.5 hc63931a_1 conda-forge proj4 6.1.1 hc80f0dc_1 conda-forge prompt-toolkit 2.0.9 pypi_0 pypi protobuf 3.9.1 pypi_0 pypi psycopg2-binary 2.8.3 pypi_0 pypi pthread-stubs 0.4 h14c3975_1001 conda-forge ptyprocess 0.6.0 pypi_0 pypi pyasn1 0.4.6 pypi_0 pypi pyasn1-modules 0.2.6 pypi_0 pypi pycosat 0.6.3 py37h14c3975_1001 conda-forge pycparser 2.19 py37_1 conda-forge pygments 2.4.2 pypi_0 pypi pyopenssl 19.0.0 py37_0 conda-forge pyphen 0.9.5 pypi_0 pypi pyproj 2.3.0 py37h2fd02e8_0 conda-forge pysocks 1.7.0 py37_0 conda-forge python 3.7.3 h33d41f4_1 conda-forge python-dateutil 2.8.0 py_0 conda-forge python-libarchive-c 2.8 py37_1004 conda-forge python-slugify 3.0.3 pypi_0 pypi pytz 2019.2 py_0 conda-forge pytzdata 2019.2 pypi_0 pypi readline 8.0 hf8c457e_0 conda-forge requests 2.22.0 py37_1 conda-forge rsa 4.0 pypi_0 pypi rtree 0.8.3 py37h666c49c_1002 conda-forge ruamel_yaml 0.15.71 py37h14c3975_1000 conda-forge sentry-sdk 0.11.1 pypi_0 pypi setuptools 41.2.0 py37_0 conda-forge shapely 1.6.4 py37hec07ddf_1006 conda-forge six 1.12.0 py37_1000 conda-forge sqlalchemy 1.3.8 pypi_0 pypi sqlite 3.29.0 hcee41ef_0 conda-forge text-unidecode 1.2 pypi_0 pypi tinycss2 1.0.2 pypi_0 pypi tk 8.6.9 hed695b0_1002 conda-forge tqdm 4.35.0 py_0 conda-forge traitlets 4.3.2 pypi_0 pypi tzcode 2019a h516909a_1002 conda-forge urllib3 1.25.3 py37_0 conda-forge utm 0.5.0 pypi_0 pypi wcwidth 0.1.7 pypi_0 pypi weasyprint 48 pypi_0 pypi webencodings 0.5.1 pypi_0 pypi wheel 0.33.6 py37_0 conda-forge xerces-c 3.2.2 hea5cb30_1003 conda-forge xorg-kbproto 1.0.7 h14c3975_1002 conda-forge xorg-libice 1.0.10 h516909a_0 conda-forge xorg-libsm 1.2.3 h84519dc_1000 conda-forge xorg-libx11 1.6.8 h516909a_0 conda-forge xorg-libxau 1.0.9 h14c3975_0 conda-forge xorg-libxdmcp 1.1.3 h516909a_0 conda-forge xorg-libxext 1.3.4 h516909a_0 conda-forge xorg-libxrender 0.9.10 h516909a_1002 conda-forge xorg-renderproto 0.11.1 h14c3975_1002 conda-forge xorg-xextproto 7.3.0 h14c3975_1002 conda-forge xorg-xproto 7.0.31 h14c3975_1007 conda-forge xz 5.2.4 h14c3975_1001 conda-forge yaml 0.1.7 h14c3975_1001 conda-forge zlib 1.2.11 h516909a_1005 conda-forge zstd 1.4.0 h3b9ef0a_0 conda-forge ```


Details about conda and system ( conda info ):

``` active environment : None user config file : /root/.condarc populated config files : /root/.condarc conda version : 4.7.11 conda-build version : not installed python version : 3.7.3.final.0 virtual packages : base environment : /opt/conda (writable) channel URLs : https://conda.anaconda.org/conda-forge/linux-64 https://conda.anaconda.org/conda-forge/noarch https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /opt/conda/pkgs /root/.conda/pkgs envs directories : /opt/conda/envs /root/.conda/envs platform : linux-64 user-agent : conda/4.7.11 requests/2.22.0 CPython/3.7.3 Linux/4.15.0-1037-gcp ubuntu/16.04.6 glibc/2.23 UID:GID : 0:0 netrc file : None offline mode : False ```
hyperknot commented 5 years ago

So I couldn't figure out what to downgrade, conda didn't even allow me to use geopandas == 0.5.0, even though it was the previous version I was using.

So I had downgraded to a few month old docker baseimage, which of course works perfectly.

Here are the details from that image, maybe that helps.


Environment (conda list):

``` # packages in environment at /opt/conda: # # Name Version Build Channel asn1crypto 0.24.0 py37_1003 conda-forge atk 2.25.90 hb9dd440_1002 conda-forge attrs 19.1.0 py_0 conda-forge backcall 0.1.0 pypi_0 pypi boost-cpp 1.68.0 h11c811c_1000 conda-forge bottleneck 1.2.1 py37h3010b51_1001 conda-forge bzip2 1.0.6 h14c3975_1002 conda-forge ca-certificates 2019.3.9 hecc5488_0 conda-forge cachetools 3.1.0 pypi_0 pypi cairo 1.16.0 ha4e643d_1000 conda-forge cairocffi 1.0.2 pypi_0 pypi cairosvg 2.4.1 pypi_0 pypi certifi 2019.3.9 py37_0 conda-forge cffi 1.12.3 py37h8022711_0 conda-forge chardet 3.0.4 py37_1003 conda-forge click 7.0 py_0 conda-forge click-plugins 1.1.1 py_0 conda-forge cligj 0.5.0 py_0 conda-forge conda 4.6.14 py37_0 conda-forge cryptography 2.5 py37hb7f436b_1 conda-forge cssselect2 0.2.1 pypi_0 pypi curl 7.64.0 h646f8bb_0 conda-forge cycler 0.10.0 py_1 conda-forge dbus 1.13.6 he372182_0 conda-forge decorator 4.4.0 pypi_0 pypi defusedxml 0.6.0 pypi_0 pypi descartes 1.1.0 py_3 conda-forge expat 2.2.5 hf484d3e_1002 conda-forge fiona 1.8.6 py37hf242f0b_0 conda-forge fontconfig 2.13.1 he4413a7_1000 conda-forge freetype 2.10.0 he983fc9_0 conda-forge freexl 1.0.5 h14c3975_1002 conda-forge gdal 2.4.0 py37h1c6dbfb_1002 conda-forge gdk-pixbuf 2.36.12 h49783d7_1002 conda-forge geoalchemy2 0.6.2 pypi_0 pypi geographiclib 1.49 pypi_0 pypi geojson 2.4.1 pypi_0 pypi geopandas 0.5.0 py_2 conda-forge geopy 1.19.0 pypi_0 pypi geos 3.7.1 hf484d3e_1000 conda-forge geotiff 1.4.3 h1105359_1000 conda-forge gettext 0.19.8.1 hc5be6a0_1002 conda-forge giflib 5.1.9 h516909a_0 conda-forge glib 2.58.3 hf63aee3_1001 conda-forge gobject-introspection 1.58.2 py37h2da5eee_1000 conda-forge google-api-core 1.10.0 pypi_0 pypi google-auth 1.6.3 pypi_0 pypi google-cloud-core 0.29.1 pypi_0 pypi google-cloud-firestore 0.32.1 pypi_0 pypi google-cloud-storage 1.15.0 pypi_0 pypi google-resumable-media 0.3.2 pypi_0 pypi googleapis-common-protos 1.5.9 pypi_0 pypi graphite2 1.3.13 hf484d3e_1000 conda-forge grpcio 1.20.1 pypi_0 pypi gstreamer 1.14.4 h66beb1c_1001 conda-forge gtk2 2.24.31 hb68c50a_1001 conda-forge harfbuzz 2.4.0 h37c48d4_0 conda-forge hdf4 4.2.13 h9a582f1_1002 conda-forge hdf5 1.10.4 nompi_h3c11f04_1106 conda-forge html5lib 1.0.1 pypi_0 pypi icu 58.2 hf484d3e_1000 conda-forge idna 2.8 py37_1000 conda-forge ipython 7.5.0 pypi_0 pypi ipython-genutils 0.2.0 pypi_0 pypi jedi 0.13.3 pypi_0 pypi jpeg 9c h14c3975_1001 conda-forge json-c 0.13.1 h14c3975_1001 conda-forge kealib 1.4.10 h1978553_1003 conda-forge kiwisolver 1.1.0 py37hc9558a2_0 conda-forge krb5 1.16.3 hc83ff2d_1000 conda-forge libblas 3.8.0 7_openblas conda-forge libcblas 3.8.0 7_openblas conda-forge libcurl 7.64.0 h01ee5af_0 conda-forge libdap4 3.19.1 hd48c02d_1000 conda-forge libedit 3.1.20170329 hf8c457e_1001 conda-forge libffi 3.2.1 he1b5a44_1006 conda-forge libgcc-ng 8.2.0 hdf63c60_1 libgdal 2.4.0 h982c1cc_1002 conda-forge libgfortran-ng 7.3.0 hdf63c60_0 libiconv 1.15 h516909a_1005 conda-forge libkml 1.3.0 h328b03d_1009 conda-forge liblapack 3.8.0 7_openblas conda-forge libnetcdf 4.6.2 hbdf4f91_1001 conda-forge libpng 1.6.37 hed695b0_0 conda-forge libpq 10.6 h13b8bad_1000 conda-forge libspatialindex 1.9.0 he1b5a44_0 conda-forge libspatialite 4.3.0a hb5ec416_1026 conda-forge libssh2 1.8.0 h1ad7b7a_1003 conda-forge libstdcxx-ng 8.2.0 hdf63c60_1 libtiff 4.0.10 h9022e91_1002 conda-forge libuuid 2.32.1 h14c3975_1000 conda-forge libxcb 1.13 h14c3975_1002 conda-forge libxml2 2.9.9 h13577e0_0 conda-forge mako 1.1.0 pypi_0 pypi mapclassify 2.0.1 py_0 conda-forge markupsafe 1.1.1 pypi_0 pypi matplotlib 3.0.3 py37_1 conda-forge matplotlib-base 3.0.3 py37h5f35d83_1 conda-forge munch 2.3.2 py_0 conda-forge ncurses 6.1 hf484d3e_1002 conda-forge numexpr 2.6.9 py37h637b7d7_1000 conda-forge numpy 1.16.3 py37he5ce36f_0 conda-forge openblas 0.3.5 h9ac9557_1001 conda-forge openjpeg 2.3.1 h58a6597_0 conda-forge openssl 1.0.2r h14c3975_0 conda-forge pandas 0.24.2 py37hf484d3e_0 conda-forge pango 1.40.14 h4ea9474_1004 conda-forge parso 0.4.0 pypi_0 pypi pcre 8.41 hf484d3e_1003 conda-forge pendulum 2.0.4 pypi_0 pypi pexpect 4.7.0 pypi_0 pypi pickleshare 0.7.5 pypi_0 pypi pillow 6.1.0 pypi_0 pypi pip 19.1 py37_0 conda-forge pixman 0.34.0 h14c3975_1003 conda-forge poppler 0.67.0 h2ad7f00_6 conda-forge poppler-data 0.4.9 1 conda-forge postgresql 10.6 h66cca7a_1000 conda-forge proj4 5.2.0 h14c3975_1001 conda-forge prompt-toolkit 2.0.9 pypi_0 pypi protobuf 3.7.1 pypi_0 pypi psycopg2-binary 2.8.2 pypi_0 pypi pthread-stubs 0.4 h14c3975_1001 conda-forge ptyprocess 0.6.0 pypi_0 pypi pyasn1 0.4.5 pypi_0 pypi pyasn1-modules 0.2.5 pypi_0 pypi pycosat 0.6.3 py37h14c3975_1001 conda-forge pycparser 2.19 py37_1 conda-forge pygments 2.3.1 pypi_0 pypi pyopenssl 19.0.0 py37_0 conda-forge pyparsing 2.4.0 py_0 conda-forge pyphen 0.9.5 pypi_0 pypi pyproj 1.9.6 py37hc0953d3_1000 conda-forge pyqt 5.6.0 py37h13b7fb3_1008 conda-forge pysocks 1.6.8 py37_1002 conda-forge python 3.7.1 hd21baee_1001 conda-forge python-dateutil 2.8.0 py_0 conda-forge python-slugify 3.0.2 pypi_0 pypi pytz 2019.1 py_0 conda-forge pytzdata 2019.1 pypi_0 pypi qt 5.6.2 hce4f676_1013 conda-forge readline 7.0 hf8c457e_1001 conda-forge requests 2.21.0 py37_1000 conda-forge rsa 4.0 pypi_0 pypi rtree 0.8.3 py37h666c49c_1002 conda-forge ruamel_yaml 0.15.71 py37h14c3975_1000 conda-forge scipy 1.2.1 py37h09a28d5_1 conda-forge sentry-sdk 0.11.1 pypi_0 pypi setuptools 41.0.1 py37_0 conda-forge shapely 1.6.4 py37h2afed24_1004 conda-forge sip 4.18.1 py37hf484d3e_1000 conda-forge six 1.12.0 py37_1000 conda-forge sqlalchemy 1.3.8 pypi_0 pypi sqlite 3.26.0 h67949de_1001 conda-forge text-unidecode 1.2 pypi_0 pypi tinycss2 1.0.2 pypi_0 pypi tk 8.6.9 h84994c4_1001 conda-forge tornado 6.0.2 py37h516909a_0 conda-forge traitlets 4.3.2 pypi_0 pypi tzcode 2018g h14c3975_1001 conda-forge urllib3 1.24.2 py37_0 conda-forge utm 0.4.2 pypi_0 pypi wcwidth 0.1.7 pypi_0 pypi weasyprint 48 pypi_0 pypi webencodings 0.5.1 pypi_0 pypi wheel 0.33.1 py37_0 conda-forge xerces-c 3.2.2 hac72e42_1001 conda-forge xorg-kbproto 1.0.7 h14c3975_1002 conda-forge xorg-libice 1.0.9 h516909a_1004 conda-forge xorg-libsm 1.2.3 h84519dc_1000 conda-forge xorg-libx11 1.6.7 h14c3975_1000 conda-forge xorg-libxau 1.0.9 h14c3975_0 conda-forge xorg-libxdmcp 1.1.3 h516909a_0 conda-forge xorg-libxext 1.3.4 h516909a_0 conda-forge xorg-libxrender 0.9.10 h516909a_1002 conda-forge xorg-libxt 1.1.5 h14c3975_1002 conda-forge xorg-renderproto 0.11.1 h14c3975_1002 conda-forge xorg-xextproto 7.3.0 h14c3975_1002 conda-forge xorg-xproto 7.0.31 h14c3975_1007 conda-forge xz 5.2.4 h14c3975_1001 conda-forge yaml 0.1.7 h14c3975_1001 conda-forge zlib 1.2.11 h14c3975_1004 conda-forge zstd 1.3.3 1 conda-forge ```


Details about conda and system ( conda info ):

``` active environment : None user config file : /root/.condarc populated config files : /root/.condarc conda version : 4.6.14 conda-build version : not installed python version : 3.7.1.final.0 base environment : /opt/conda (writable) channel URLs : https://conda.anaconda.org/conda-forge/linux-64 https://conda.anaconda.org/conda-forge/noarch https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/free/linux-64 https://repo.anaconda.com/pkgs/free/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /opt/conda/pkgs /root/.conda/pkgs envs directories : /opt/conda/envs /root/.conda/envs platform : linux-64 user-agent : conda/4.6.14 requests/2.21.0 CPython/3.7.1 Linux/4.15.0-1037-gcp ubuntu/16.04.6 glibc/2.23 UID:GID : 0:0 netrc file : None offline mode : False ```
ocefpaf commented 5 years ago

Your image mixes packages from defaults, pypi and conda-forge. It is virtually impossible for us to debug that. If you can re-create an environment using strict channel and conda-forge on top of defaults we can probably help.

Closing this b/c we cannot debug it.

hyperknot commented 5 years ago

Those are unrelated packages, mostly for web development. But case in point, here is a minimal reproducible Dockerfile:

FROM ubuntu:16.04

RUN apt-get update && apt-get install -y wget

RUN wget --quiet https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh && \
    /bin/bash ~/miniconda.sh -b -p /opt/conda && \
    /opt/conda/bin/conda update conda -y && \
    /opt/conda/bin/conda config --add channels conda-forge && \
    /opt/conda/bin/conda config --set channel_priority strict && \
    /opt/conda/bin/conda install \
        geopandas numexpr bottleneck

RUN /opt/conda/bin/python -c 'import fiona'
``` Sending build context to Docker daemon 4.608kB Step 1/4 : FROM ubuntu:16.04 ---> 5e13f8dd4c1a Step 2/4 : RUN apt-get update && apt-get install -y wget ---> Using cache ---> 1810ea1a30a0 Step 3/4 : RUN wget --quiet https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh && /bin/bash ~/miniconda.sh -b -p /opt/conda && /opt/conda/bin/conda update conda -y && /opt/conda/bin/conda config --add channels conda-forge && /opt/conda/bin/conda config --set channel_priority strict && /opt/conda/bin/conda install geopandas numexpr bottleneck ---> Running in abeb1c40b5bb PREFIX=/opt/conda Unpacking payload ... Collecting package metadata (current_repodata.json): ...working... done Solving environment: ...working... done ## Package Plan ## environment location: /opt/conda added / updated specs: - _libgcc_mutex==0.1=main - asn1crypto==0.24.0=py37_0 - bzip2==1.0.8=h7b6447c_0 - ca-certificates==2019.5.15=0 - certifi==2019.6.16=py37_0 - cffi==1.12.3=py37h2e261b9_0 - chardet==3.0.4=py37_1 - conda-package-handling==1.3.11=py37_0 - conda==4.7.10=py37_0 - cryptography==2.7=py37h1ba5d50_0 - idna==2.8=py37_0 - libarchive==3.3.3=h5d8350f_5 - libedit==3.1.20181209=hc058e9b_0 - libffi==3.2.1=hd88cf55_4 - libgcc-ng==9.1.0=hdf63c60_0 - libstdcxx-ng==9.1.0=hdf63c60_0 - libxml2==2.9.9=hea5a465_1 - lz4-c==1.8.1.2=h14c3975_0 - lzo==2.10=h49e0be7_2 - ncurses==6.1=he6710b0_1 - openssl==1.1.1c=h7b6447c_1 - pip==19.1.1=py37_0 - pycosat==0.6.3=py37h14c3975_0 - pycparser==2.19=py37_0 - pyopenssl==19.0.0=py37_0 - pysocks==1.7.0=py37_0 - python-libarchive-c==2.8=py37_11 - python==3.7.3=h0371630_0 - readline==7.0=h7b6447c_5 - requests==2.22.0=py37_0 - ruamel_yaml==0.15.46=py37h14c3975_0 - setuptools==41.0.1=py37_0 - six==1.12.0=py37_0 - sqlite==3.29.0=h7b6447c_0 - tk==8.6.8=hbc83047_0 - tqdm==4.32.1=py_0 - urllib3==1.24.2=py37_0 - wheel==0.33.4=py37_0 - xz==5.2.4=h14c3975_4 - yaml==0.1.7=had09818_2 - zlib==1.2.11=h7b6447c_3 - zstd==1.3.7=h0b5b093_0 The following NEW packages will be INSTALLED: _libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main asn1crypto pkgs/main/linux-64::asn1crypto-0.24.0-py37_0 bzip2 pkgs/main/linux-64::bzip2-1.0.8-h7b6447c_0 ca-certificates pkgs/main/linux-64::ca-certificates-2019.5.15-0 certifi pkgs/main/linux-64::certifi-2019.6.16-py37_0 cffi pkgs/main/linux-64::cffi-1.12.3-py37h2e261b9_0 chardet pkgs/main/linux-64::chardet-3.0.4-py37_1 conda pkgs/main/linux-64::conda-4.7.10-py37_0 conda-package-han~ pkgs/main/linux-64::conda-package-handling-1.3.11-py37_0 cryptography pkgs/main/linux-64::cryptography-2.7-py37h1ba5d50_0 idna pkgs/main/linux-64::idna-2.8-py37_0 libarchive pkgs/main/linux-64::libarchive-3.3.3-h5d8350f_5 libedit pkgs/main/linux-64::libedit-3.1.20181209-hc058e9b_0 libffi pkgs/main/linux-64::libffi-3.2.1-hd88cf55_4 libgcc-ng pkgs/main/linux-64::libgcc-ng-9.1.0-hdf63c60_0 libstdcxx-ng pkgs/main/linux-64::libstdcxx-ng-9.1.0-hdf63c60_0 libxml2 pkgs/main/linux-64::libxml2-2.9.9-hea5a465_1 lz4-c pkgs/main/linux-64::lz4-c-1.8.1.2-h14c3975_0 lzo pkgs/main/linux-64::lzo-2.10-h49e0be7_2 ncurses pkgs/main/linux-64::ncurses-6.1-he6710b0_1 openssl pkgs/main/linux-64::openssl-1.1.1c-h7b6447c_1 pip pkgs/main/linux-64::pip-19.1.1-py37_0 pycosat pkgs/main/linux-64::pycosat-0.6.3-py37h14c3975_0 pycparser pkgs/main/linux-64::pycparser-2.19-py37_0 pyopenssl pkgs/main/linux-64::pyopenssl-19.0.0-py37_0 pysocks pkgs/main/linux-64::pysocks-1.7.0-py37_0 python pkgs/main/linux-64::python-3.7.3-h0371630_0 python-libarchive~ pkgs/main/linux-64::python-libarchive-c-2.8-py37_11 readline pkgs/main/linux-64::readline-7.0-h7b6447c_5 requests pkgs/main/linux-64::requests-2.22.0-py37_0 ruamel_yaml pkgs/main/linux-64::ruamel_yaml-0.15.46-py37h14c3975_0 setuptools pkgs/main/linux-64::setuptools-41.0.1-py37_0 six pkgs/main/linux-64::six-1.12.0-py37_0 sqlite pkgs/main/linux-64::sqlite-3.29.0-h7b6447c_0 tk pkgs/main/linux-64::tk-8.6.8-hbc83047_0 tqdm pkgs/main/noarch::tqdm-4.32.1-py_0 urllib3 pkgs/main/linux-64::urllib3-1.24.2-py37_0 wheel pkgs/main/linux-64::wheel-0.33.4-py37_0 xz pkgs/main/linux-64::xz-5.2.4-h14c3975_4 yaml pkgs/main/linux-64::yaml-0.1.7-had09818_2 zlib pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3 zstd pkgs/main/linux-64::zstd-1.3.7-h0b5b093_0 Preparing transaction: ...working... done Executing transaction: ...working... WARNING conda.core.envs_manager:register_env(46): Unable to register environment. Path not writable or missing. environment location: /opt/conda registry file: /root/.conda/environments.txt done installation finished. Collecting package metadata (current_repodata.json): ...working... done Solving environment: ...working... done ## Package Plan ## environment location: /opt/conda added / updated specs: - conda The following packages will be downloaded: package | build ---------------------------|----------------- ca-certificates-2019.5.15 | 1 134 KB certifi-2019.6.16 | py37_1 156 KB chardet-3.0.4 | py37_1003 173 KB conda-4.7.11 | py37_0 3.0 MB pip-19.2.2 | py37_0 1.9 MB python-libarchive-c-2.8 | py37_13 23 KB ------------------------------------------------------------ Total: 5.4 MB The following packages will be UPDATED: ca-certificates 2019.5.15-0 --> 2019.5.15-1 certifi 2019.6.16-py37_0 --> 2019.6.16-py37_1 chardet 3.0.4-py37_1 --> 3.0.4-py37_1003 conda 4.7.10-py37_0 --> 4.7.11-py37_0 pip 19.1.1-py37_0 --> 19.2.2-py37_0 python-libarchive~ 2.8-py37_11 --> 2.8-py37_13 Downloading and Extracting Packages chardet-3.0.4 | 173 KB | ########## | 100% certifi-2019.6.16 | 156 KB | ########## | 100% ca-certificates-2019 | 134 KB | ########## | 100% pip-19.2.2 | 1.9 MB | ########## | 100% python-libarchive-c- | 23 KB | ########## | 100% conda-4.7.11 | 3.0 MB | ########## | 100% Preparing transaction: ...working... done Verifying transaction: ...working... done Executing transaction: ...working... done Collecting package metadata (current_repodata.json): ...working... done Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve. Solving environment: ...working... done Collecting package metadata (repodata.json): ...working... done Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve. Solving environment: ...working... done ## Package Plan ## environment location: /opt/conda added / updated specs: - bottleneck - geopandas - numexpr The following packages will be downloaded: package | build ---------------------------|----------------- asn1crypto-0.24.0 | py37_1003 154 KB conda-forge attrs-19.1.0 | py_0 32 KB conda-forge boost-cpp-1.70.0 | ha2d47e9_1 21.1 MB conda-forge bottleneck-1.2.1 |py37h3010b51_1001 127 KB conda-forge bzip2-1.0.8 | h516909a_0 396 KB conda-forge ca-certificates-2019.6.16 | hecc5488_0 145 KB conda-forge cairo-1.16.0 | h18b612c_1001 1.5 MB conda-forge certifi-2019.6.16 | py37_1 149 KB conda-forge cffi-1.12.3 | py37h8022711_0 218 KB conda-forge cfitsio-3.470 | hb60a0a2_2 1.4 MB conda-forge chardet-3.0.4 | py37_1003 167 KB conda-forge click-7.0 | py_0 61 KB conda-forge click-plugins-1.1.1 | py_0 9 KB conda-forge cligj-0.5.0 | py_0 8 KB conda-forge conda-4.7.11 | py37_0 3.0 MB conda-forge conda-package-handling-1.4.1| py37_0 262 KB conda-forge cryptography-2.7 | py37h72c5cf5_0 610 KB conda-forge curl-7.65.3 | hf8cf82a_0 132 KB conda-forge expat-2.2.5 | he1b5a44_1003 191 KB conda-forge fiona-1.8.6 | py37h5f563d9_4 1.1 MB conda-forge fontconfig-2.13.1 | he4413a7_1000 327 KB conda-forge freetype-2.10.0 | he983fc9_1 884 KB conda-forge freexl-1.0.5 | h14c3975_1002 43 KB conda-forge gdal-2.4.2 | py37h0b7157c_7 1.3 MB conda-forge geopandas-0.5.1 | py_0 887 KB conda-forge geos-3.7.2 | he1b5a44_1 872 KB conda-forge geotiff-1.5.1 | hcd53e25_3 279 KB conda-forge gettext-0.19.8.1 | hc5be6a0_1002 3.6 MB conda-forge giflib-5.1.7 | h516909a_1 455 KB conda-forge glib-2.58.3 | h6f030ca_1002 3.3 MB conda-forge hdf4-4.2.13 | h9a582f1_1002 952 KB conda-forge hdf5-1.10.4 |nompi_h3c11f04_1106 5.3 MB conda-forge icu-58.2 | hf484d3e_1000 22.6 MB conda-forge idna-2.8 | py37_1000 100 KB conda-forge jpeg-9c | h14c3975_1001 251 KB conda-forge json-c-0.13.1 | h14c3975_1001 71 KB conda-forge kealib-1.4.10 | h1978553_1003 172 KB conda-forge krb5-1.16.3 | h05b26f9_1001 1.4 MB conda-forge libarchive-3.3.3 | hb44662c_1005 1.4 MB conda-forge libblas-3.8.0 | 12_openblas 10 KB conda-forge libcblas-3.8.0 | 12_openblas 10 KB conda-forge libcurl-7.65.3 | hda55be3_0 575 KB conda-forge libdap4-3.20.2 | hd48c02d_1000 18.6 MB conda-forge libedit-3.1.20170329 | hf8c457e_1001 172 KB conda-forge libffi-3.2.1 | he1b5a44_1006 46 KB conda-forge libgdal-2.4.2 | h182c168_7 18.6 MB conda-forge libgfortran-ng-7.3.0 | hdf63c60_0 1006 KB libiconv-1.15 | h516909a_1005 2.0 MB conda-forge libkml-1.3.0 | h4fcabce_1010 643 KB conda-forge liblapack-3.8.0 | 12_openblas 10 KB conda-forge libnetcdf-4.6.2 | hbdf4f91_1001 1.3 MB conda-forge libopenblas-0.3.7 | h6e990d7_1 7.6 MB conda-forge libpng-1.6.37 | hed695b0_0 343 KB conda-forge libpq-11.5 | hd9ab2ff_1 2.5 MB conda-forge libspatialindex-1.9.0 | he1b5a44_1 3.1 MB conda-forge libspatialite-4.3.0a | h57ae47a_1030 3.1 MB conda-forge libssh2-1.8.2 | h22169c7_2 257 KB conda-forge libtiff-4.0.10 | h57b8799_1003 587 KB conda-forge libuuid-2.32.1 | h14c3975_1000 26 KB conda-forge libxcb-1.13 | h14c3975_1002 396 KB conda-forge libxml2-2.9.9 | h13577e0_2 1.3 MB conda-forge lz4-c-1.8.3 | he1b5a44_1001 187 KB conda-forge lzo-2.10 | h14c3975_1000 319 KB conda-forge munch-2.3.2 | py_0 8 KB conda-forge ncurses-6.1 | hf484d3e_1002 1.3 MB conda-forge numexpr-2.7.0 | py37hb3f55d8_0 197 KB conda-forge numpy-1.17.1 | py37h95a1406_0 5.2 MB conda-forge openjpeg-2.3.1 | h58a6597_0 470 KB conda-forge openssl-1.1.1c | h516909a_0 2.1 MB conda-forge pandas-0.25.1 | py37hb3f55d8_0 11.4 MB conda-forge pcre-8.41 | hf484d3e_1003 249 KB conda-forge pip-19.2.3 | py37_0 1.9 MB conda-forge pixman-0.38.0 | h516909a_1003 594 KB conda-forge poppler-0.67.0 | ha967d66_7 8.9 MB conda-forge poppler-data-0.4.9 | 1 3.4 MB conda-forge postgresql-11.5 | hc63931a_1 4.8 MB conda-forge proj4-6.1.1 | hc80f0dc_1 9.8 MB conda-forge pthread-stubs-0.4 | h14c3975_1001 5 KB conda-forge pycosat-0.6.3 |py37h14c3975_1001 104 KB conda-forge pycparser-2.19 | py37_1 171 KB conda-forge pyopenssl-19.0.0 | py37_0 81 KB conda-forge pyproj-2.3.0 | py37h2fd02e8_0 269 KB conda-forge pysocks-1.7.0 | py37_0 26 KB conda-forge python-3.7.3 | h33d41f4_1 36.0 MB conda-forge python-dateutil-2.8.0 | py_0 219 KB conda-forge python-libarchive-c-2.8 | py37_1004 21 KB conda-forge pytz-2019.2 | py_0 228 KB conda-forge readline-8.0 | hf8c457e_0 441 KB conda-forge requests-2.22.0 | py37_1 84 KB conda-forge rtree-0.8.3 |py37h666c49c_1002 48 KB conda-forge ruamel_yaml-0.15.71 |py37h14c3975_1000 256 KB conda-forge setuptools-41.2.0 | py37_0 635 KB conda-forge shapely-1.6.4 |py37hec07ddf_1006 339 KB conda-forge six-1.12.0 | py37_1000 22 KB conda-forge sqlite-3.29.0 | hcee41ef_1 1.9 MB conda-forge tk-8.6.9 | hed695b0_1002 3.2 MB conda-forge tqdm-4.35.0 | py_0 42 KB conda-forge tzcode-2019a | h516909a_1002 429 KB conda-forge urllib3-1.25.3 | py37_0 187 KB conda-forge wheel-0.33.6 | py37_0 35 KB conda-forge xerces-c-3.2.2 | hea5cb30_1003 1.7 MB conda-forge xorg-kbproto-1.0.7 | h14c3975_1002 26 KB conda-forge xorg-libice-1.0.10 | h516909a_0 57 KB conda-forge xorg-libsm-1.2.3 | h84519dc_1000 25 KB conda-forge xorg-libx11-1.6.8 | h516909a_0 907 KB conda-forge xorg-libxau-1.0.9 | h14c3975_0 13 KB conda-forge xorg-libxdmcp-1.1.3 | h516909a_0 18 KB conda-forge xorg-libxext-1.3.4 | h516909a_0 51 KB conda-forge xorg-libxrender-0.9.10 | h516909a_1002 31 KB conda-forge xorg-renderproto-0.11.1 | h14c3975_1002 8 KB conda-forge xorg-xextproto-7.3.0 | h14c3975_1002 27 KB conda-forge xorg-xproto-7.0.31 | h14c3975_1007 72 KB conda-forge xz-5.2.4 | h14c3975_1001 366 KB conda-forge yaml-0.1.7 | h14c3975_1001 78 KB conda-forge zlib-1.2.11 | h516909a_1005 105 KB conda-forge zstd-1.4.0 | h3b9ef0a_0 928 KB conda-forge ------------------------------------------------------------ Total: 236.7 MB The following NEW packages will be INSTALLED: attrs conda-forge/noarch::attrs-19.1.0-py_0 boost-cpp conda-forge/linux-64::boost-cpp-1.70.0-ha2d47e9_1 bottleneck conda-forge/linux-64::bottleneck-1.2.1-py37h3010b51_1001 cairo conda-forge/linux-64::cairo-1.16.0-h18b612c_1001 cfitsio conda-forge/linux-64::cfitsio-3.470-hb60a0a2_2 click conda-forge/noarch::click-7.0-py_0 click-plugins conda-forge/noarch::click-plugins-1.1.1-py_0 cligj conda-forge/noarch::cligj-0.5.0-py_0 curl conda-forge/linux-64::curl-7.65.3-hf8cf82a_0 expat conda-forge/linux-64::expat-2.2.5-he1b5a44_1003 fiona conda-forge/linux-64::fiona-1.8.6-py37h5f563d9_4 fontconfig conda-forge/linux-64::fontconfig-2.13.1-he4413a7_1000 freetype conda-forge/linux-64::freetype-2.10.0-he983fc9_1 freexl conda-forge/linux-64::freexl-1.0.5-h14c3975_1002 gdal conda-forge/linux-64::gdal-2.4.2-py37h0b7157c_7 geopandas conda-forge/noarch::geopandas-0.5.1-py_0 geos conda-forge/linux-64::geos-3.7.2-he1b5a44_1 geotiff conda-forge/linux-64::geotiff-1.5.1-hcd53e25_3 gettext conda-forge/linux-64::gettext-0.19.8.1-hc5be6a0_1002 giflib conda-forge/linux-64::giflib-5.1.7-h516909a_1 glib conda-forge/linux-64::glib-2.58.3-h6f030ca_1002 hdf4 conda-forge/linux-64::hdf4-4.2.13-h9a582f1_1002 hdf5 conda-forge/linux-64::hdf5-1.10.4-nompi_h3c11f04_1106 icu conda-forge/linux-64::icu-58.2-hf484d3e_1000 jpeg conda-forge/linux-64::jpeg-9c-h14c3975_1001 json-c conda-forge/linux-64::json-c-0.13.1-h14c3975_1001 kealib conda-forge/linux-64::kealib-1.4.10-h1978553_1003 krb5 conda-forge/linux-64::krb5-1.16.3-h05b26f9_1001 libblas conda-forge/linux-64::libblas-3.8.0-12_openblas libcblas conda-forge/linux-64::libcblas-3.8.0-12_openblas libcurl conda-forge/linux-64::libcurl-7.65.3-hda55be3_0 libdap4 conda-forge/linux-64::libdap4-3.20.2-hd48c02d_1000 libgdal conda-forge/linux-64::libgdal-2.4.2-h182c168_7 libgfortran-ng pkgs/main/linux-64::libgfortran-ng-7.3.0-hdf63c60_0 libiconv conda-forge/linux-64::libiconv-1.15-h516909a_1005 libkml conda-forge/linux-64::libkml-1.3.0-h4fcabce_1010 liblapack conda-forge/linux-64::liblapack-3.8.0-12_openblas libnetcdf conda-forge/linux-64::libnetcdf-4.6.2-hbdf4f91_1001 libopenblas conda-forge/linux-64::libopenblas-0.3.7-h6e990d7_1 libpng conda-forge/linux-64::libpng-1.6.37-hed695b0_0 libpq conda-forge/linux-64::libpq-11.5-hd9ab2ff_1 libspatialindex conda-forge/linux-64::libspatialindex-1.9.0-he1b5a44_1 libspatialite conda-forge/linux-64::libspatialite-4.3.0a-h57ae47a_1030 libssh2 conda-forge/linux-64::libssh2-1.8.2-h22169c7_2 libtiff conda-forge/linux-64::libtiff-4.0.10-h57b8799_1003 libuuid conda-forge/linux-64::libuuid-2.32.1-h14c3975_1000 libxcb conda-forge/linux-64::libxcb-1.13-h14c3975_1002 munch conda-forge/noarch::munch-2.3.2-py_0 numexpr conda-forge/linux-64::numexpr-2.7.0-py37hb3f55d8_0 numpy conda-forge/linux-64::numpy-1.17.1-py37h95a1406_0 openjpeg conda-forge/linux-64::openjpeg-2.3.1-h58a6597_0 pandas conda-forge/linux-64::pandas-0.25.1-py37hb3f55d8_0 pcre conda-forge/linux-64::pcre-8.41-hf484d3e_1003 pixman conda-forge/linux-64::pixman-0.38.0-h516909a_1003 poppler conda-forge/linux-64::poppler-0.67.0-ha967d66_7 poppler-data conda-forge/noarch::poppler-data-0.4.9-1 postgresql conda-forge/linux-64::postgresql-11.5-hc63931a_1 proj4 conda-forge/linux-64::proj4-6.1.1-hc80f0dc_1 pthread-stubs conda-forge/linux-64::pthread-stubs-0.4-h14c3975_1001 pyproj conda-forge/linux-64::pyproj-2.3.0-py37h2fd02e8_0 python-dateutil conda-forge/noarch::python-dateutil-2.8.0-py_0 pytz conda-forge/noarch::pytz-2019.2-py_0 rtree conda-forge/linux-64::rtree-0.8.3-py37h666c49c_1002 shapely conda-forge/linux-64::shapely-1.6.4-py37hec07ddf_1006 tzcode conda-forge/linux-64::tzcode-2019a-h516909a_1002 xerces-c conda-forge/linux-64::xerces-c-3.2.2-hea5cb30_1003 xorg-kbproto conda-forge/linux-64::xorg-kbproto-1.0.7-h14c3975_1002 xorg-libice conda-forge/linux-64::xorg-libice-1.0.10-h516909a_0 xorg-libsm conda-forge/linux-64::xorg-libsm-1.2.3-h84519dc_1000 xorg-libx11 conda-forge/linux-64::xorg-libx11-1.6.8-h516909a_0 xorg-libxau conda-forge/linux-64::xorg-libxau-1.0.9-h14c3975_0 xorg-libxdmcp conda-forge/linux-64::xorg-libxdmcp-1.1.3-h516909a_0 xorg-libxext conda-forge/linux-64::xorg-libxext-1.3.4-h516909a_0 xorg-libxrender conda-forge/linux-64::xorg-libxrender-0.9.10-h516909a_1002 xorg-renderproto conda-forge/linux-64::xorg-renderproto-0.11.1-h14c3975_1002 xorg-xextproto conda-forge/linux-64::xorg-xextproto-7.3.0-h14c3975_1002 xorg-xproto conda-forge/linux-64::xorg-xproto-7.0.31-h14c3975_1007 The following packages will be UPDATED: asn1crypto pkgs/main::asn1crypto-0.24.0-py37_0 --> conda-forge::asn1crypto-0.24.0-py37_1003 ca-certificates pkgs/main::ca-certificates-2019.5.15-1 --> conda-forge::ca-certificates-2019.6.16-hecc5488_0 conda-package-han~ pkgs/main::conda-package-handling-1.3~ --> conda-forge::conda-package-handling-1.4.1-py37_0 idna pkgs/main::idna-2.8-py37_0 --> conda-forge::idna-2.8-py37_1000 libarchive pkgs/main::libarchive-3.3.3-h5d8350f_5 --> conda-forge::libarchive-3.3.3-hb44662c_1005 libffi pkgs/main::libffi-3.2.1-hd88cf55_4 --> conda-forge::libffi-3.2.1-he1b5a44_1006 libxml2 pkgs/main::libxml2-2.9.9-hea5a465_1 --> conda-forge::libxml2-2.9.9-h13577e0_2 lz4-c pkgs/main::lz4-c-1.8.1.2-h14c3975_0 --> conda-forge::lz4-c-1.8.3-he1b5a44_1001 lzo pkgs/main::lzo-2.10-h49e0be7_2 --> conda-forge::lzo-2.10-h14c3975_1000 ncurses pkgs/main::ncurses-6.1-he6710b0_1 --> conda-forge::ncurses-6.1-hf484d3e_1002 pip pkgs/main::pip-19.2.2-py37_0 --> conda-forge::pip-19.2.3-py37_0 pycosat pkgs/main::pycosat-0.6.3-py37h14c3975~ --> conda-forge::pycosat-0.6.3-py37h14c3975_1001 pycparser pkgs/main::pycparser-2.19-py37_0 --> conda-forge::pycparser-2.19-py37_1 python pkgs/main::python-3.7.3-h0371630_0 --> conda-forge::python-3.7.3-h33d41f4_1 python-libarchive~ pkgs/main::python-libarchive-c-2.8-py~ --> conda-forge::python-libarchive-c-2.8-py37_1004 readline pkgs/main::readline-7.0-h7b6447c_5 --> conda-forge::readline-8.0-hf8c457e_0 requests pkgs/main::requests-2.22.0-py37_0 --> conda-forge::requests-2.22.0-py37_1 ruamel_yaml pkgs/main::ruamel_yaml-0.15.46-py37h1~ --> conda-forge::ruamel_yaml-0.15.71-py37h14c3975_1000 setuptools pkgs/main::setuptools-41.0.1-py37_0 --> conda-forge::setuptools-41.2.0-py37_0 six pkgs/main::six-1.12.0-py37_0 --> conda-forge::six-1.12.0-py37_1000 sqlite pkgs/main::sqlite-3.29.0-h7b6447c_0 --> conda-forge::sqlite-3.29.0-hcee41ef_1 tk pkgs/main::tk-8.6.8-hbc83047_0 --> conda-forge::tk-8.6.9-hed695b0_1002 tqdm pkgs/main::tqdm-4.32.1-py_0 --> conda-forge::tqdm-4.35.0-py_0 urllib3 pkgs/main::urllib3-1.24.2-py37_0 --> conda-forge::urllib3-1.25.3-py37_0 wheel pkgs/main::wheel-0.33.4-py37_0 --> conda-forge::wheel-0.33.6-py37_0 xz pkgs/main::xz-5.2.4-h14c3975_4 --> conda-forge::xz-5.2.4-h14c3975_1001 yaml pkgs/main::yaml-0.1.7-had09818_2 --> conda-forge::yaml-0.1.7-h14c3975_1001 zlib pkgs/main::zlib-1.2.11-h7b6447c_3 --> conda-forge::zlib-1.2.11-h516909a_1005 zstd pkgs/main::zstd-1.3.7-h0b5b093_0 --> conda-forge::zstd-1.4.0-h3b9ef0a_0 The following packages will be SUPERSEDED by a higher-priority channel: bzip2 pkgs/main::bzip2-1.0.8-h7b6447c_0 --> conda-forge::bzip2-1.0.8-h516909a_0 certifi pkgs/main --> conda-forge cffi pkgs/main::cffi-1.12.3-py37h2e261b9_0 --> conda-forge::cffi-1.12.3-py37h8022711_0 chardet pkgs/main --> conda-forge conda pkgs/main --> conda-forge cryptography pkgs/main::cryptography-2.7-py37h1ba5~ --> conda-forge::cryptography-2.7-py37h72c5cf5_0 libedit pkgs/main::libedit-3.1.20181209-hc058~ --> conda-forge::libedit-3.1.20170329-hf8c457e_1001 openssl pkgs/main::openssl-1.1.1c-h7b6447c_1 --> conda-forge::openssl-1.1.1c-h516909a_0 pyopenssl pkgs/main --> conda-forge pysocks pkgs/main --> conda-forge Proceed ([y]/n)? Downloading and Extracting Packages xorg-xextproto-7.3.0 | 27 KB | ########## | 100% boost-cpp-1.70.0 | 21.1 MB | ########## | 100% libspatialite-4.3.0a | 3.1 MB | ########## | 100% conda-4.7.11 | 3.0 MB | ########## | 100% conda-package-handli | 262 KB | ########## | 100% fontconfig-2.13.1 | 327 KB | ########## | 100% libxcb-1.13 | 396 KB | ########## | 100% click-plugins-1.1.1 | 9 KB | ########## | 100% pip-19.2.3 | 1.9 MB | ########## | 100% cffi-1.12.3 | 218 KB | ########## | 100% fiona-1.8.6 | 1.1 MB | ########## | 100% lzo-2.10 | 319 KB | ########## | 100% liblapack-3.8.0 | 10 KB | ########## | 100% libcurl-7.65.3 | 575 KB | ########## | 100% poppler-0.67.0 | 8.9 MB | ########## | 100% xorg-renderproto-0.1 | 8 KB | ########## | 100% icu-58.2 | 22.6 MB | ########## | 100% wheel-0.33.6 | 35 KB | ########## | 100% tqdm-4.35.0 | 42 KB | ########## | 100% shapely-1.6.4 | 339 KB | ########## | 100% asn1crypto-0.24.0 | 154 KB | ########## | 100% xorg-libxdmcp-1.1.3 | 18 KB | ########## | 100% zlib-1.2.11 | 105 KB | ########## | 100% munch-2.3.2 | 8 KB | ########## | 100% requests-2.22.0 | 84 KB | ########## | 100% libgfortran-ng-7.3.0 | 1006 KB | ########## | 100% ruamel_yaml-0.15.71 | 256 KB | ########## | 100% click-7.0 | 61 KB | ########## | 100% idna-2.8 | 100 KB | ########## | 100% libtiff-4.0.10 | 587 KB | ########## | 100% geotiff-1.5.1 | 279 KB | ########## | 100% python-dateutil-2.8. | 219 KB | ########## | 100% libiconv-1.15 | 2.0 MB | ########## | 100% xorg-libxau-1.0.9 | 13 KB | ########## | 100% pthread-stubs-0.4 | 5 KB | ########## | 100% hdf5-1.10.4 | 5.3 MB | ########## | 100% xorg-libsm-1.2.3 | 25 KB | ########## | 100% hdf4-4.2.13 | 952 KB | ########## | 100% poppler-data-0.4.9 | 3.4 MB | ########## | 100% attrs-19.1.0 | 32 KB | ########## | 100% krb5-1.16.3 | 1.4 MB | ########## | 100% postgresql-11.5 | 4.8 MB | ########## | 100% libpng-1.6.37 | 343 KB | ########## | 100% sqlite-3.29.0 | 1.9 MB | ########## | 100% json-c-0.13.1 | 71 KB | ########## | 100% xz-5.2.4 | 366 KB | ########## | 100% openjpeg-2.3.1 | 470 KB | ########## | 100% libuuid-2.32.1 | 26 KB | ########## | 100% numexpr-2.7.0 | 197 KB | ########## | 100% freexl-1.0.5 | 43 KB | ########## | 100% bottleneck-1.2.1 | 127 KB | ########## | 100% geos-3.7.2 | 872 KB | ########## | 100% ncurses-6.1 | 1.3 MB | ########## | 100% jpeg-9c | 251 KB | ########## | 100% libopenblas-0.3.7 | 7.6 MB | ########## | 100% python-libarchive-c- | 21 KB | ########## | 100% pcre-8.41 | 249 KB | ########## | 100% tk-8.6.9 | 3.2 MB | ########## | 100% cryptography-2.7 | 610 KB | ########## | 100% certifi-2019.6.16 | 149 KB | ########## | 100% pycparser-2.19 | 171 KB | ########## | 100% lz4-c-1.8.3 | 187 KB | ########## | 100% libblas-3.8.0 | 10 KB | ########## | 100% freetype-2.10.0 | 884 KB | ########## | 100% python-3.7.3 | 36.0 MB | ########## | 100% curl-7.65.3 | 132 KB | ########## | 100% pytz-2019.2 | 228 KB | ########## | 100% pixman-0.38.0 | 594 KB | ########## | 100% libdap4-3.20.2 | 18.6 MB | ########## | 100% tzcode-2019a | 429 KB | ########## | 100% urllib3-1.25.3 | 187 KB | ########## | 100% libedit-3.1.20170329 | 172 KB | ########## | 100% libgdal-2.4.2 | 18.6 MB | ########## | 100% gdal-2.4.2 | 1.3 MB | ########## | 100% giflib-5.1.7 | 455 KB | ########## | 100% glib-2.58.3 | 3.3 MB | ########## | 100% libspatialindex-1.9. | 3.1 MB | ########## | 100% bzip2-1.0.8 | 396 KB | ########## | 100% xorg-kbproto-1.0.7 | 26 KB | ########## | 100% xerces-c-3.2.2 | 1.7 MB | ########## | 100% libcblas-3.8.0 | 10 KB | ########## | 100% gettext-0.19.8.1 | 3.6 MB | ########## | 100% xorg-libxrender-0.9. | 31 KB | ########## | 100% zstd-1.4.0 | 928 KB | ########## | 100% pyproj-2.3.0 | 269 KB | ########## | 100% ca-certificates-2019 | 145 KB | ########## | 100% rtree-0.8.3 | 48 KB | ########## | 100% setuptools-41.2.0 | 635 KB | ########## | 100% xorg-libice-1.0.10 | 57 KB | ########## | 100% pysocks-1.7.0 | 26 KB | ########## | 100% libkml-1.3.0 | 643 KB | ########## | 100% pycosat-0.6.3 | 104 KB | ########## | 100% libpq-11.5 | 2.5 MB | ########## | 100% openssl-1.1.1c | 2.1 MB | ########## | 100% readline-8.0 | 441 KB | ########## | 100% cligj-0.5.0 | 8 KB | ########## | 100% cairo-1.16.0 | 1.5 MB | ########## | 100% geopandas-0.5.1 | 887 KB | ########## | 100% xorg-xproto-7.0.31 | 72 KB | ########## | 100% libxml2-2.9.9 | 1.3 MB | ########## | 100% libffi-3.2.1 | 46 KB | ########## | 100% libarchive-3.3.3 | 1.4 MB | ########## | 100% chardet-3.0.4 | 167 KB | ########## | 100% six-1.12.0 | 22 KB | ########## | 100% libnetcdf-4.6.2 | 1.3 MB | ########## | 100% yaml-0.1.7 | 78 KB | ########## | 100% numpy-1.17.1 | 5.2 MB | ########## | 100% expat-2.2.5 | 191 KB | ########## | 100% cfitsio-3.470 | 1.4 MB | ########## | 100% xorg-libx11-1.6.8 | 907 KB | ########## | 100% proj4-6.1.1 | 9.8 MB | ########## | 100% kealib-1.4.10 | 172 KB | ########## | 100% pyopenssl-19.0.0 | 81 KB | ########## | 100% xorg-libxext-1.3.4 | 51 KB | ########## | 100% pandas-0.25.1 | 11.4 MB | ########## | 100% libssh2-1.8.2 | 257 KB | ########## | 100% Preparing transaction: ...working... done Verifying transaction: ...working... done Executing transaction: ...working... done Removing intermediate container abeb1c40b5bb ---> 822807ddfaad Step 4/4 : RUN /opt/conda/bin/python -c 'import fiona' ---> Running in 5bd3c0e4b80d proj_create: Open of /opt/conda/share/proj failed proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode proj_create: Open of /opt/conda/share/proj failed proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode ```
hyperknot commented 5 years ago

In https://github.com/pyproj4/pyproj/issues/415#issuecomment-525979663, @snowman2 figured out that the reason for this is that global environment works differently from local. I don't know why is this the case, I was always using global environment in Docker images and I'd like to continue doing that.

So if I have run conda create -n env geopandas numexpr bottleneck the import errors, and I believe the segfaults are not present.

hyperknot commented 5 years ago

I've created a super minimal repro case:

Dockerfile

FROM ubuntu:16.04

RUN apt-get update && apt-get install -y wget

RUN wget --quiet https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh && \
    /bin/bash ~/miniconda.sh -b -p /opt/conda && \
    /opt/conda/bin/conda update conda -y && \
    /opt/conda/bin/conda config --add channels conda-forge && \
    /opt/conda/bin/conda config --set channel_priority strict

RUN /opt/conda/bin/conda install \
        geopandas numexpr bottleneck

RUN /opt/conda/bin/python -c 'import fiona'

RUN /opt/conda/bin/python -c 'import pyproj; pyproj.show_versions()'

ADD test.py .
RUN /opt/conda/bin/python -X faulthandler test.py

test.py

import geopandas
import pandas as pd
from shapely.geometry import Point

d = {'lat': [48], 'lon': [17]}
df = pd.DataFrame(data=d)

df['Coordinates'] = list(zip(df.lon, df.lat))
df['Coordinates'] = df['Coordinates'].apply(Point)

gdf = geopandas.GeoDataFrame(df, geometry='Coordinates', crs={'init': 'epsg:4326'})

gdf = gdf.to_crs('+proj=utm +zone=33 +ellps=WGS84 +datum=WGS84 +units=m +no_defs')

The output is:

Step 5/8 : RUN /opt/conda/bin/python -c 'import fiona'
 ---> Running in 5244dc7a9907
proj_create: Open of /opt/conda/share/proj failed
proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode
proj_create: Open of /opt/conda/share/proj failed
proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode
Removing intermediate container 5244dc7a9907
 ---> fbb3cea55d37
Step 6/8 : RUN /opt/conda/bin/python -c 'import pyproj; pyproj.show_versions()'
 ---> Running in e08c7e848abf

System:
    python: 3.7.3 | packaged by conda-forge | (default, Jul  1 2019, 21:52:21)  [GCC 7.3.0]
executable: /opt/conda/bin/python
   machine: Linux-4.15.0-1037-gcp-x86_64-with-debian-stretch-sid

PROJ:
      PROJ: 6.1.1
  data dir: /opt/conda/share/proj

Python deps:
    pyproj: 2.3.0
       pip: 19.2.3
setuptools: 41.2.0
    Cython: None
Removing intermediate container e08c7e848abf
 ---> cde69b9050bc
Step 7/8 : ADD test.py .
 ---> 910f70195298
Step 8/8 : RUN /opt/conda/bin/python test.py
 ---> Running in febd45a3ac0a
proj_create: Open of /opt/conda/share/proj failed
proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode
proj_create: Open of /opt/conda/share/proj failed
proj_create: init=epsg:/init=IGNF: syntax not supported in non-PROJ4 emulation mode
Fatal Python error: Segmentation fault

Current thread 0x00007fa8eb545700 (most recent call first):
  File "/opt/conda/lib/python3.7/site-packages/pyproj/crs.py", line 303 in __init__
  File "/opt/conda/lib/python3.7/site-packages/pyproj/crs.py", line 434 in from_user_input
  File "/opt/conda/lib/python3.7/site-packages/pyproj/proj.py", line 145 in __init__
  File "/opt/conda/lib/python3.7/site-packages/geopandas/geoseries.py", line 304 in to_crs
  File "/opt/conda/lib/python3.7/site-packages/geopandas/geodataframe.py", line 459 in to_crs
  File "test.py", line 13 in <module>
Segmentation fault (core dumped)
The command '/bin/sh -c /opt/conda/bin/python -X faulthandler test.py' returned a non-zero code: 139
hyperknot commented 5 years ago

Updated ^^^ with python -X faulthandler

hyperknot commented 5 years ago

I believe I've reached the core of the problem in https://github.com/pyproj4/pyproj/issues/415.

I can conclude there are exactly two bugs here:

  1. If PROJ_LIB is not found then pyproj 2.3.0 Segfaults, instead of terminating with a message like 2.2 does. This should be fixed in pyproj core, nothing to do with conda I believe.
  2. There is a problem with PROJ_LIB variable in newer Conda install. That effect Fiona, etc. as well.
    • if I add PROJ_LIB=/opt/conda/share/proj, both Fiona and Pyproj works like before.
    • if I pip uninstall and pip install pyproj, so that it's installed from pip instead of Conda then it works perfectly without PROJ_LIB. Same for fiona.

So Conda install of pyproj and fiona are broken and pip install works.

RUN /opt/conda/bin/pip uninstall -y pyproj
RUN /opt/conda/bin/pip install pyproj

RUN /opt/conda/bin/pip uninstall -y fiona
RUN /opt/conda/bin/pip install fiona
ocefpaf commented 5 years ago

There is a problem with PROJ_LIB variable in newer Conda install. That effect Fiona, etc. as well.

What problem? It works fine when I create an environment and activate it as conda is designed to work.

hyperknot commented 5 years ago

Yes, but it breaks with the global install. In a docker environment it's common to use global install, as there is no bash environment to speak of.

On 2019. Aug 29., Thu at 17:46, Filipe notifications@github.com wrote:

There is a problem with PROJ_LIB variable in newer Conda install. That effect Fiona, etc. as well.

What problem? It works fine when I create an environment and activate it as conda is designed to work.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/conda-forge/geopandas-feedstock/issues/63?email_source=notifications&email_token=AADYVDYLEBBDYQTYTXI2VWLQG7VNJA5CNFSM4IRV53K2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5O6DCQ#issuecomment-526246282, or mute the thread https://github.com/notifications/unsubscribe-auth/AADYVD5ZYBRBLMHQ3SQSGMLQG7VNJANCNFSM4IRV53KQ .

ocefpaf commented 5 years ago

Yes, but it breaks with the global install. In a docker environment it's common to use global install, as there is no bash environment to speak of.

That is a misuse of the conda design. It is not a problem with the package. You can configure the base env though. You should do something like conda init and then conda activate base. I'm locking this issue b/c it is not related to geopandas or conda-forge packages. It is a conda on docker questions.