search

conda-forge / fiona-feedstock

A conda-smithy repository for fiona.
BSD 3-Clause "New" or "Revised" License
8 stars 17 forks source link

Circular import errors on Windows #192

Closed adamjstewart closed 2 years ago

adamjstewart commented 2 years ago

Solution to issue cannot be found in the documentation.

Issue

We've been encountering the following issue (https://github.com/Toblerity/Fiona/issues/944) in our Windows CI since yesterday's rebuild (https://github.com/conda-forge/fiona-feedstock/pull/191). See https://github.com/microsoft/torchgeo/runs/5710380368 for the exact issue.

@weiji14 @theroggy

Installed packages

# packages in environment at C:\Miniconda\envs\test:
#
# Name                    Version                   Build  Channel
affine                    2.3.1              pyhd8ed1ab_0    conda-forge
attrs                     21.4.0             pyhd8ed1ab_0    conda-forge
blosc                     1.21.0               h0e60522_0    conda-forge
boost-cpp                 1.74.0               h9f4b32c_7    conda-forge
bzip2                     1.0.8                h8ffe710_4    conda-forge
ca-certificates           2021.10.8            h5b45459_0    conda-forge
cairo                     1.16.0            h15b3021_1010    conda-forge
certifi                   2021.10.8        py39hcbf5309_1    conda-forge
cfitsio                   4.1.0                h5a969a9_0    conda-forge
click                     8.0.4            py39hcbf5309_0    conda-forge
click-plugins             1.1.1                      py_0    conda-forge
cligj                     0.7.2              pyhd8ed1ab_1    conda-forge
colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
curl                      7.82.0               h789b8ee_0    conda-forge
expat                     2.4.7                h39d44d4_0    conda-forge
fiona                     1.8.21           py39hd99abff_1    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.13.96              hce3cb01_2    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
freetype                  2.10.4               h546665d_1    conda-forge
freexl                    1.0.6                ha8e266a_0    conda-forge
gdal                      3.4.2            py39hd395a97_2    conda-forge
geos                      3.10.2               h39d44d4_0    conda-forge
geotiff                   1.7.0                h38b14a8_7    conda-forge
gettext                   0.19.8.1          ha2e2712_1008    conda-forge
hdf4                      4.2.15               h0e5069d_3    conda-forge
hdf5                      1.12.1          nompi_h57737ce_104    conda-forge
icu                       69.1                 h0e60522_0    conda-forge
intel-openmp              2022.0.0          h57928b3_3663    conda-forge
jbig                      2.1               h8d14728_2003    conda-forge
jpeg                      9e                   h8ffe710_0    conda-forge
kealib                    1.4.14               h8995ca9_3    conda-forge
krb5                      1.19.3               hc8ab02b_0    conda-forge
lcms2                     2.12                 h2a16943_0    conda-forge
lerc                      3.0                  h0e60522_0    conda-forge
libblas                   3.9.0              13_win64_mkl    conda-forge
libcblas                  3.9.0              13_win64_mkl    conda-forge
libcurl                   7.82.0               h789b8ee_0    conda-forge
libdeflate                1.10                 h8ffe710_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libgdal                   3.4.2                hf817787_2    conda-forge
libglib                   2.70.2               h3be07f2_4    conda-forge
libiconv                  1.16                 he774522_0    conda-forge
libkml                    1.3.0             h9859afa_1014    conda-forge
liblapack                 3.9.0              13_win64_mkl    conda-forge
libnetcdf                 4.8.1           nompi_h1cc8e9d_101    conda-forge
libpng                    1.6.37               h1d00b33_2    conda-forge
libpq                     14.2                 h1ea2d34_0    conda-forge
librttopo                 1.1.0                hb1df466_9    conda-forge
libspatialite             5.0.1               h36c16d9_15    conda-forge
libssh2                   1.10.0               h9a1e1f7_2    conda-forge
libtiff                   4.3.0                hc4061b1_3    conda-forge
libwebp-base              1.2.2                h8ffe710_1    conda-forge
libxml2                   2.9.12               hf5bbc77_1    conda-forge
libzip                    1.8.0                h519de47_1    conda-forge
libzlib                   1.2.11            h8ffe710_1014    conda-forge
lz4-c                     1.9.3                h8ffe710_1    conda-forge
mkl                       2022.0.0           h0e2418a_796    conda-forge
munch                     2.5.0                      py_0    conda-forge
numpy                     1.22.3           py39h6331f09_0    conda-forge
openjpeg                  2.4.0                hb211442_1    conda-forge
openssl                   3.0.2                h8ffe710_1    conda-forge
pcre                      8.45                 h0e60522_0    conda-forge
pip                       22.0.4             pyhd8ed1ab_0    conda-forge
pixman                    0.40.0               h8ffe710_0    conda-forge
poppler                   22.01.0              h24fffdf_2    conda-forge
poppler-data              0.4.11               hd8ed1ab_0    conda-forge
postgresql                14.2                 he353ca9_0    conda-forge
proj                      9.0.0                h1cfcee9_1    conda-forge
pyparsing                 3.0.7              pyhd8ed1ab_0    conda-forge
python                    3.9.12          hcf16a7b_1_cpython    conda-forge
python_abi                3.9                      2_cp39    conda-forge
rasterio                  1.2.10           py39h0f7fabe_5    conda-forge
setuptools                61.1.1           py39hcbf5309_0    conda-forge
shapely                   1.8.0            py39hf063a39_5    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
snuggs                    1.4.7                      py_0    conda-forge
sqlite                    3.37.1               h8ffe710_0    conda-forge
tbb                       2021.5.0             h2d74725_0    conda-forge
tiledb                    2.7.1                h47404fa_0    conda-forge
tk                        8.6.12               h8ffe710_0    conda-forge
tzdata                    2022a                h191b570_0    conda-forge
ucrt                      10.0.20348.0         h57928b3_0    conda-forge
vc                        14.2                 hb210afc_6    conda-forge
vs2015_runtime            14.29.30037          h902a5da_6    conda-forge
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xerces-c                  3.2.3                h0e60522_4    conda-forge
xz                        5.2.5                h62dcd97_1    conda-forge
zlib                      1.2.11            h8ffe710_1014    conda-forge
zstd                      1.5.2                h6255e5f_0    conda-forge

Environment info

active environment : test
    active env location : C:\Miniconda\envs\test
            shell level : 2
       user config file : C:\Users\runneradmin\.condarc
 populated config files : C:\Users\runneradmin\.condarc
          conda version : 4.11.0
    conda-build version : not installed
         python version : 3.9.7.final.0
       virtual packages : __win=0=0
                          __archspec=1=x86_64
       base environment : C:\Miniconda  (writable)
      conda av data dir : C:\Miniconda\etc\conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/win-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : C:\Users\runneradmin\conda_pkgs_dir
       envs directories : C:\Miniconda\envs
                          C:\Users\runneradmin\.conda\envs
                          C:\Users\runneradmin\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/4.11.0 requests/2.27.1 CPython/3.9.7 Windows/10 Windows/10.0.20348
          administrator : True
             netrc file : None
           offline mode : False
jorisvandenbossche commented 2 years ago

As a short-term fix, I marked the windows packages of the latest build as broken (https://github.com/conda-forge/admin-requests/pull/418), so a default install of the latest version on windows should work again.

Now we of course still need to figure out why this started to fail with the latest rerendering.

jorisvandenbossche commented 2 years ago

So I think that in the end this didn't have to do with the rebuild of fiona itself, but with a rebuild of gdal that happened at almost the same time.

You can see see here in one of the failure reports that also the gdal build number was bumped when this started failing: https://github.com/kedro-org/kedro/issues/1377#issuecomment-1080905543

And I was able to now test it locally on a Windows machine, and if using the latest gdal build, the "broken" fiona build is also working fine. But then when installing the previous gdal build (gdal 3.4.2 build 2, https://anaconda.org/conda-forge/libgdal/files), it starts to fail.

And the reason this started working again is probably also not because of marking the fiona build as broken, but just that more or less at the same time, it got fixed in gdal: https://github.com/conda-forge/geotiff-feedstock/pull/47 (bumping build number of libgdal to 3), https://github.com/conda-forge/gdal-feedstock/issues/616, https://github.com/conda-forge/geotiff-feedstock/pull/47

akrherz commented 2 years ago

There's now a new GDAL build 4 for version 3.4.2. I am unsure what to suggest be tried here...

jorisvandenbossche commented 2 years ago

You mean that the error re-appeared with the new gdal build?

akrherz commented 2 years ago

You mean that the error re-appeared with the new gdal build?

I have not tried the new build to see if it has the same troubles as other recent GDAL builds have had.

jorisvandenbossche commented 2 years ago

Since there have been no reports (and our own tests are passing on https://github.com/conda-forge/fiona-feedstock/pull/195), I think we can assume it is still working.

Closing this issue, as there is nothing left to do / all should be resolved.