conda-forge / fiona-feedstock

A conda-smithy repository for fiona.
BSD 3-Clause "New" or "Revised" License
8 stars 17 forks source link

ImportError: DLL load failed while importing _env (Windows) #213

Closed Sieboldianus closed 8 months ago

Sieboldianus commented 1 year ago

Solution to issue cannot be found in the documentation.

Issue

I have a package with fiona that (still) needs to be installed in Windows.

The error appears to be related to gdal and the conda installaton of fiona. I could reproduce it both with the anaconda distribution and conda-forge channel. The error is raised by fiona, so I came here to report it.

When gdal and fiona are installed in versions larger than fiona 1.8.22, the following occurs:

Traceback (most recent call last): File "", line 1, in File "C:\tools\miniconda3\envs\fiona_test\Lib\site-packages\fiona__init__.py", line 86, in from fiona._env import ( ImportError: DLL load failed while importing _env: The specified procedure could not be found.

I tried with different combinations, channels and versions.

First, adding:

conda env config vars set USE_PATH_FOR_GDAL_PYTHON=YES

does not solve the error.

Further:

gdalinfo --version

works in all combinations, except when gdal 3.4.1 is installed, then this happens:

Click ![image](https://github.com/conda-forge/gdal-feedstock/assets/13646666/44bdad9f-caa8-4f49-bd5e-e9d3728ed3c3)

I think this is not related and can be ignored.

Installed packages

Minimal steps to preproduce:


conda create -n fiona_test gdal fiona --yes
conda activate fiona_test
python -c "import fiona;print(fiona.__version__);"

Traceback (most recent call last): File "", line 1, in File "C:\tools\miniconda3\envs\fiona_test\Lib\site-packages\fiona__init__.py", line 86, in from fiona._env import ( ImportError: DLL load failed while importing _env: The specified procedure could not be found.

conda list -n fiona_test

-> gdal 3.6.2 -> fiona 1.9.1

conda install "fiona<1.9.1"
conda create -n fiona_test3 gdal "fiona<1.9.1" --yes
conda activate fiona_test3
python -c "import fiona;print(fiona.__version__);"

-> works -> gdal 3.0.2 -> fiona 1.8.22

Conda-forge:

conda create -n fiona_test2 -c conda-forge "gdal>3.4.1" "fiona>=1.8.22" --yes
conda activate fiona_test2
python -c "import fiona;print(fiona.__version__);"

-> does not work -> gdal-3.6.4 -> fiona-1.9.3

conda create -n fiona_test4 -c conda-forge gdal "fiona<1.9.1" --yes
conda activate fiona_test4
python -c "import fiona;print(fiona.__version__);"

Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source. -> does not work -> gdal-3.6.4 -> fiona-1.9.0

conda create -n fiona_test5 -c conda-forge "gdal<=3.6.1" fiona --yes
conda activate fiona_test5
python -c "import fiona;print(fiona.__version__);"

-> works -> gdal-3.6.1 -> fiona-1.8.22

Environment info

I have Miniconda installed with Chocolatey. The tests above have been run in plain Windows CMD.

     active environment : fiona_test4
    active env location : C:\tools\miniconda3\envs\fiona_test4
            shell level : 2
       user config file : C:\Users\AD\.condarc
 populated config files : C:\Users\AD\.condarc
          conda version : 23.3.1
    conda-build version : not installed
         python version : 3.9.12.final.0
       virtual packages : __archspec=1=x86_64
                          __cuda=11.7=0
                          __win=0=0
       base environment : C:\tools\miniconda3  (writable)
      conda av data dir : C:\tools\miniconda3\etc\conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : C:\tools\miniconda3\pkgs
                          C:\Users\AD\.conda\pkgs
                          C:\Users\AD\AppData\Local\conda\conda\pkgs
       envs directories : C:\tools\miniconda3\envs
                          C:\Users\AD\.conda\envs
                          C:\Users\AD\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/23.3.1 requests/2.29.0 CPython/3.9.12 Windows/10 Windows/10.0.19045
          administrator : True
             netrc file : None
           offline mode : False
akrherz commented 9 months ago

@Sieboldianus , sorry that you didn't get a response to this. Fixing windows build troubles is always painful to will the spare energy to look into :/ Are you still reproducing this? I tried just now and failed to.

$ mamba create -n dev fiona
$ conda activate dev
$ python -c "import fiona;print(fiona.__version__);"
1.9.4
Sieboldianus commented 9 months ago

Hi @akrherz , thank you, and I completely understand (re: Windows).

I tried your exact steps, except that I used conda instead of mamba, and could still reproduce my issue:

$ conda create -n dev fiona
$ conda activate dev
$ python -c "import fiona;print(fiona.__version__);"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\tools\Miniconda3\envs\dev\Lib\site-packages\fiona\__init__.py", line 86, in <module>
    from fiona._env import (
ImportError: DLL load failed while importing _env: The specified module could not be found.
MischaelR commented 9 months ago

I have been facing a similar issue over the past month. My solution was to use pyogrio instead of fiona for reading shapefiles within geopandas.

That said, I am still having the issue with fiona. Documenting here in case it helps. I am also using conda instead of mamba.

image

image

Similar issue within the terminal.

image

I don't have much experience with these things, but happy to help troubleshoot if it is useful.

akrherz commented 9 months ago

Thanks, can we ensure that we have:

  1. conda-forge in your channel URLS? conda info
  2. You have strict channel priority set conda config --show channel_priority
Sieboldianus commented 9 months ago

Ok, so first: I just tried with a fresh mambaforge installation (chocolatey), and this time it worked:

$ mamba create -n dev fiona
$ mamba activate dev
$ python -c "import fiona;print(fiona.__version__);"
1.9.4

This is my conda info: (I don't have conda-forge in my channel urls, as I usually use -c conda-forge - but the results are the same)

conda info
>     active environment : None
       user config file : C:\Users\AD\.condarc
 populated config files :
          conda version : 22.9.0
    conda-build version : not installed
         python version : 3.7.4.final.0
       virtual packages : __cuda=12.0=0
                          __win=0=0
                          __archspec=1=x86_64
       base environment : C:\tools\Miniconda3  (writable)
      conda av data dir : C:\tools\Miniconda3\etc\conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : C:\tools\Miniconda3\pkgs
                          C:\Users\AD\.conda\pkgs
                          C:\Users\AD\AppData\Local\conda\conda\pkgs
       envs directories : C:\tools\Miniconda3\envs
                          C:\Users\AD\.conda\envs
                          C:\Users\AD\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/22.9.0 requests/2.28.1 CPython/3.7.4 Windows/10 Windows/10.0.19041
          administrator : False
             netrc file : None
           offline mode : False

My channel priority is set to flexible:

conda config --show channel_priority
> channel_priority: flexible

I also reproduced this issue on two different computers.

akrherz commented 9 months ago

On a reproducer, what does conda list report?

Sieboldianus commented 9 months ago
conda activate dev
conda list
> # packages in environment at C:\tools\Miniconda3\envs\dev:
#
# Name                    Version                   Build  Channel
attrs                     22.1.0          py311haa95532_0
blas                      1.0                         mkl
blosc                     1.21.3               h6c2663c_0
boost-cpp                 1.82.0               h59b6b97_1
bzip2                     1.0.8                he774522_0
ca-certificates           2023.08.22           haa95532_0
cairo                     1.16.0               haedb8bc_5
certifi                   2023.7.22       py311haa95532_0
cfitsio                   3.470                h2bbff1b_7
click                     8.0.4           py311haa95532_0
click-plugins             1.1.1              pyhd3eb1b0_0
cligj                     0.7.2              pyhd3eb1b0_0
colorama                  0.4.6           py311haa95532_0
expat                     2.5.0                hd77b12b_0
fiona                     1.9.1           py311heda8569_0
fontconfig                2.14.1               h9c4af85_2
freetype                  2.12.1               ha860e81_0
freexl                    1.0.6                h2bbff1b_0
gdal                      3.6.2           py311h4e7b5b2_3
geos                      3.8.0                h33f27b4_0
geotiff                   1.7.0                h4545760_1
glib                      2.69.1               h5dc1a3c_2
hdf4                      4.2.13               h712560f_2
hdf5                      1.12.1               h51c971a_3
icc_rt                    2022.1.0             h6049295_2
intel-openmp              2023.1.0         h59b6b97_46319
jpeg                      9e                   h2bbff1b_1
kealib                    1.5.0                hde4a422_1
lcms2                     2.12                 h83e58a3_0
lerc                      3.0                  hd77b12b_0
libboost                  1.82.0               hae598e9_1
libcurl                   8.1.1                h86230a5_0
libdeflate                1.17                 h2bbff1b_0
libffi                    3.4.4                hd77b12b_0
libgdal                   3.6.2                h676b171_3
libiconv                  1.16                 h2bbff1b_2
libkml                    1.3.0                h63940dd_7
libnetcdf                 4.8.1                h6685c40_4
libpng                    1.6.39               h8cc25b3_0
libpq                     12.15                h906ac69_0
libspatialite             4.3.0a              h6ec8781_23
libssh2                   1.10.0               he2ea4bf_2
libtiff                   4.5.1                hd77b12b_0
libwebp-base              1.3.2                h2bbff1b_0
libxml2                   2.10.4               h0ad7f3c_1
libzip                    1.8.0                h289538f_1
lz4-c                     1.9.4                h2bbff1b_0
mkl                       2023.1.0         h6b88ed4_46357
mkl-service               2.4.0           py311h2bbff1b_1
mkl_fft                   1.3.8           py311h2bbff1b_0
mkl_random                1.2.4           py311h59b6b97_0
munch                     2.5.0              pyhd3eb1b0_0
numpy                     1.25.2          py311hdab7c0b_0
numpy-base                1.25.2          py311hd01c5d8_0
openjpeg                  2.4.0                h4fc8c34_0
openssl                   3.0.11               h2bbff1b_2
pcre                      8.45                 hd77b12b_0
pcre2                     10.42                h0ff8eda_0
pip                       23.2.1          py311haa95532_0
pixman                    0.40.0               h2bbff1b_1
poppler                   22.12.0              h268424c_0
poppler-data              0.4.11               haa95532_1
proj                      6.2.1                h3758d61_0
python                    3.11.5               he1021f5_0
qhull                     2020.2               h59b6b97_2
setuptools                68.0.0          py311haa95532_0
shapely                   2.0.1           py311h672afca_0
six                       1.16.0             pyhd3eb1b0_1
sqlite                    3.41.2               h2bbff1b_0
tbb                       2021.8.0             h59b6b97_0
tiledb                    2.3.3                hd8964de_3
tk                        8.6.12               h2bbff1b_0
tzdata                    2023c                h04d1e81_0
uriparser                 0.9.7                h2bbff1b_0
vc                        14.2                 h21ff451_1
vs2015_runtime            14.27.29016          h5e58377_2
wheel                     0.38.4          py311haa95532_0
xerces-c                  3.2.4                hd77b12b_1
xz                        5.4.2                h8cc25b3_0
zlib                      1.2.13               h8cc25b3_0
zstd                      1.5.5                hd43e919_0
akrherz commented 9 months ago

Erm, you need to be using conda-forge and not the defaults/anaconda repo.

Sieboldianus commented 9 months ago

Ok, understood - I tested, and this time, it works:

conda create -n dev3 fiona -c conda-forge
conda activate dev3 
python -c "import fiona;print(fiona.__version__);"
> 1.9.4
conda list ``` # packages in environment at C:\tools\Miniconda3\envs\dev3: # # Name Version Build Channel attrs 23.1.0 pyh71513ae_1 conda-forge blosc 1.21.5 hdccc3a2_0 conda-forge boost-cpp 1.78.0 h9f4b32c_4 conda-forge bzip2 1.0.8 h8ffe710_4 conda-forge ca-certificates 2023.7.22 h56e8100_0 conda-forge cairo 1.16.0 h412253b_1017 conda-forge cfitsio 4.3.0 h9b0cee5_0 conda-forge click 8.1.7 win_pyh7428d3b_0 conda-forge click-plugins 1.1.1 py_0 conda-forge cligj 0.7.2 pyhd8ed1ab_1 conda-forge colorama 0.4.6 pyhd8ed1ab_0 conda-forge expat 2.5.0 h63175ca_1 conda-forge fiona 1.9.4 py311h4e4dc46_0 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 hab24e00_0 conda-forge fontconfig 2.14.2 hbde0cde_0 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge freetype 2.12.1 hdaf720e_2 conda-forge freexl 2.0.0 h8276f4a_0 conda-forge gdal 3.7.2 py311h9601e46_4 conda-forge geos 3.12.0 h1537add_0 conda-forge geotiff 1.7.1 h2e186c9_13 conda-forge gettext 0.21.1 h5728263_0 conda-forge hdf4 4.2.15 h1334946_6 conda-forge hdf5 1.14.2 nompi_h73e8ff5_100 conda-forge icu 73.2 h63175ca_0 conda-forge importlib-metadata 6.8.0 pyha770c72_0 conda-forge intel-openmp 2023.2.0 h57928b3_49503 conda-forge kealib 1.5.1 ha10e780_5 conda-forge krb5 1.21.2 heb0366b_0 conda-forge lcms2 2.15 he9d350c_2 conda-forge lerc 4.0.0 h63175ca_0 conda-forge libaec 1.0.6 h63175ca_1 conda-forge libarchive 3.7.2 h6f8411a_0 conda-forge libblas 3.9.0 18_win64_mkl conda-forge libcblas 3.9.0 18_win64_mkl conda-forge libcurl 8.3.0 hd5e4a3a_0 conda-forge libdeflate 1.19 hcfcfb64_0 conda-forge libexpat 2.5.0 h63175ca_1 conda-forge libffi 3.4.2 h8ffe710_5 conda-forge libgdal 3.7.2 hc5c2e26_4 conda-forge libglib 2.78.0 he8f3873_0 conda-forge libhwloc 2.9.3 default_haede6df_1009 conda-forge libiconv 1.17 h8ffe710_0 conda-forge libjpeg-turbo 2.1.5.1 hcfcfb64_1 conda-forge libkml 1.3.0 hf2ab4e4_1015 conda-forge liblapack 3.9.0 18_win64_mkl conda-forge libnetcdf 4.9.2 nompi_h8284064_112 conda-forge libpng 1.6.39 h19919ed_0 conda-forge libpq 16.0 h43585b0_0 conda-forge librttopo 1.1.0 h92c5fdb_14 conda-forge libspatialite 5.1.0 hbf340bc_0 conda-forge libsqlite 3.43.0 hcfcfb64_0 conda-forge libssh2 1.11.0 h7dfc565_0 conda-forge libtiff 4.6.0 h4554b19_1 conda-forge libwebp-base 1.3.2 hcfcfb64_0 conda-forge libxml2 2.11.5 hc3477c8_1 conda-forge libzip 1.10.1 h1d365fa_3 conda-forge libzlib 1.2.13 hcfcfb64_5 conda-forge lz4-c 1.9.4 hcfcfb64_0 conda-forge lzo 2.10 he774522_1000 conda-forge minizip 4.0.1 h5bed578_4 conda-forge mkl 2022.1.0 h6a75c08_874 conda-forge munch 4.0.0 pyhd8ed1ab_0 conda-forge numpy 1.26.0 py311h0b4df5a_0 conda-forge openjpeg 2.5.0 h3d672ee_3 conda-forge openssl 3.1.3 hcfcfb64_0 conda-forge pcre2 10.40 h17e33f8_0 conda-forge pip 23.2.1 pyhd8ed1ab_0 conda-forge pixman 0.40.0 h8ffe710_0 conda-forge poppler 23.08.0 h35ab50b_1 conda-forge poppler-data 0.4.12 hd8ed1ab_0 conda-forge postgresql 16.0 hc80876b_0 conda-forge proj 9.3.0 he13c7e8_1 conda-forge pthreads-win32 2.9.1 hfa6e2cd_3 conda-forge python 3.11.5 h2628c8c_0_cpython conda-forge python_abi 3.11 4_cp311 conda-forge setuptools 68.2.2 pyhd8ed1ab_0 conda-forge shapely 2.0.1 py311h72efec2_3 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge snappy 1.1.10 hfb803bf_0 conda-forge sqlite 3.43.0 hcfcfb64_0 conda-forge tbb 2021.10.0 h91493d7_1 conda-forge tiledb 2.16.3 h1ffc264_3 conda-forge tk 8.6.13 hcfcfb64_0 conda-forge tzdata 2023c h71feb2d_0 conda-forge ucrt 10.0.22621.0 h57928b3_0 conda-forge vc 14.3 h64f974e_17 conda-forge vc14_runtime 14.36.32532 hdcecf7f_17 conda-forge vs2015_runtime 14.36.32532 h05e6639_17 conda-forge wheel 0.41.2 pyhd8ed1ab_0 conda-forge xerces-c 3.2.4 h63175ca_3 conda-forge xz 5.2.6 h8d14728_0 conda-forge zipp 3.17.0 pyhd8ed1ab_0 conda-forge zlib 1.2.13 hcfcfb64_5 conda-forge zstd 1.5.5 h12be248_0 conda-forge ```

In my original (initial) post, this did not work. It seems solved know!

Apologies for any confusion and many thanks for your help, @akrherz !

zklaus commented 8 months ago

This seems solved. Please reopen if that is not the case.

ocefpaf commented 8 months ago

This seems solved. Please reopen if that is not the case.

B/c similar problems can have different causes, please open a new issue instead and paste the debugging information. We should lock this one to avoid red herrings debugging.

zklaus commented 8 months ago

Ok, since the original reporter gave his thumbs up on the closing, I'll lock it.