search

conda-forge / shapely-feedstock

A conda-smithy repository for shapely.
BSD 3-Clause "New" or "Revised" License
9 stars 20 forks source link

shapely.speedups.disable() required to resolve polygon creation bug #130

Open NorahBrown opened 8 months ago

NorahBrown commented 8 months ago

Solution to issue cannot be found in the documentation.

Issue

The following code causes python to crash

Using Shapely 2.0.1 with python 3.10 on windows 10.

(ddb-api) Y:\ddb-api>python
Python 3.10.10 | packaged by conda-forge | (main, Mar 24 2023, 20:00:38) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
from shapely.geometry import box

b = box(0,1,1,2) 
# Crashes python

Using speedups.disable resolves issue

from shapely import speedups
from shapely.geometry import box

speedups.disable()
b = box(0,1,1,2)
# Works

Installed packages

# packages in environment at C:\Users\nbrown\Miniconda3\envs\ddb-api:
#
# Name                    Version                   Build  Channel
affine                    2.4.0              pyhd8ed1ab_0    conda-forge
aiofiles                  23.1.0             pyhd8ed1ab_1    conda-forge
alembic                   1.10.4             pyhd8ed1ab_0    conda-forge
annotated-types           0.5.0              pyhd8ed1ab_0    conda-forge
anyio                     3.7.1              pyhd8ed1ab_0    conda-forge
argon2-cffi               23.1.0             pyhd8ed1ab_0    conda-forge
argon2-cffi-bindings      21.2.0          py310h8d17308_3    conda-forge
asttokens                 2.2.1              pyhd8ed1ab_0    conda-forge
attrs                     23.1.0             pyh71513ae_1    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
bcrypt                    4.0.1           py310h87d50f1_0    conda-forge
blosc                     1.21.5               hdccc3a2_0    conda-forge
boost-cpp                 1.78.0               h9f4b32c_4    conda-forge
boto3                     1.28.20            pyhd8ed1ab_0    conda-forge
botocore                  1.31.41            pyhd8ed1ab_0    conda-forge
brotlipy                  0.7.0           py310h8d17308_1005    conda-forge
bzip2                     1.0.8                h8ffe710_4    conda-forge
ca-certificates           2023.7.22            h56e8100_0    conda-forge
cairo                     1.16.0            h412253b_1017    conda-forge
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
cffi                      1.15.1          py310h628cb3f_3    conda-forge
cfitsio                   4.3.0                h9b0cee5_0    conda-forge
charset-normalizer        3.2.0              pyhd8ed1ab_0    conda-forge
click                     8.1.7           win_pyh7428d3b_0    conda-forge
click-plugins             1.1.1                      py_0    conda-forge
cligj                     0.7.2              pyhd8ed1ab_1    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
cryptography              41.0.3          py310h6e82f81_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
ecdsa                     0.18.0             pyhd8ed1ab_1    conda-forge
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
executing                 1.2.0              pyhd8ed1ab_0    conda-forge
expat                     2.5.0                h63175ca_1    conda-forge
fastapi                   0.103.1            pyhd8ed1ab_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.2               hbde0cde_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
freetype                  2.12.1               h546665d_1    conda-forge
freexl                    1.0.6                h67ca5e6_1    conda-forge
geos                      3.12.0               h1537add_0    conda-forge
geotiff                   1.7.1               h4e61e90_11    conda-forge
gettext                   0.21.1               h5728263_0    conda-forge
gmpy2                     2.1.2           py310hf735c17_1    conda-forge
greenlet                  2.0.2           py310h00ffb61_1    conda-forge
h11                       0.14.0             pyhd8ed1ab_0    conda-forge
h2                        4.1.0              pyhd8ed1ab_0    conda-forge
hdf4                      4.2.15               h1334946_6    conda-forge
hdf5                      1.14.2          nompi_h73e8ff5_100    conda-forge
hpack                     4.0.0              pyh9f0ad1d_0    conda-forge
httpcore                  0.17.3             pyhd8ed1ab_0    conda-forge
httpx                     0.24.1             pyhd8ed1ab_0    conda-forge
hyperframe                6.0.1              pyhd8ed1ab_0    conda-forge
icu                       73.2                 h63175ca_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib_resources       6.0.1              pyhd8ed1ab_0    conda-forge
iniconfig                 2.0.0              pyhd8ed1ab_0    conda-forge
intel-openmp              2023.2.0         h57928b3_49496    conda-forge
ipython                   8.14.0             pyh08f2357_0    conda-forge
jedi                      0.19.0             pyhd8ed1ab_0    conda-forge
jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
jmespath                  1.0.1              pyhd8ed1ab_0    conda-forge
kealib                    1.5.1                ha10e780_5    conda-forge
krb5                      1.21.2               heb0366b_0    conda-forge
lcms2                     2.15                 h3e3b177_1    conda-forge
lerc                      4.0.0                h63175ca_0    conda-forge
libaec                    1.0.6                h63175ca_1    conda-forge
libarchive                3.6.2                h6f8411a_1    conda-forge
libblas                   3.9.0              18_win64_mkl    conda-forge
libcblas                  3.9.0              18_win64_mkl    conda-forge
libcurl                   8.2.1                hd5e4a3a_0    conda-forge
libdeflate                1.18                 hcfcfb64_0    conda-forge
libexpat                  2.5.0                h63175ca_1    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libgdal                   3.7.1                hb1fd9af_9    conda-forge
libglib                   2.76.4               he8f3873_0    conda-forge
libhwloc                  2.9.2           default_haede6df_1009    conda-forge
libiconv                  1.17                 h8ffe710_0    conda-forge
libjpeg-turbo             2.1.5.1              hcfcfb64_0    conda-forge
libkml                    1.3.0             hf2ab4e4_1015    conda-forge
liblapack                 3.9.0              18_win64_mkl    conda-forge
libnetcdf                 4.9.2           nompi_h8284064_112    conda-forge
libpng                    1.6.39               h19919ed_0    conda-forge
libpq                     15.4                 h43585b0_0    conda-forge
librttopo                 1.1.0               h92c5fdb_14    conda-forge
libspatialite             5.0.1               hc49ff46_28    conda-forge
libsqlite                 3.43.0               hcfcfb64_0    conda-forge
libssh2                   1.11.0               h7dfc565_0    conda-forge
libtiff                   4.5.1                h6c8260b_1    conda-forge
libwebp-base              1.3.1                hcfcfb64_0    conda-forge
libxml2                   2.11.5               hc3477c8_1    conda-forge
libzip                    1.10.1               h1d365fa_2    conda-forge
libzlib                   1.2.13               hcfcfb64_5    conda-forge
lz4-c                     1.9.4                hcfcfb64_0    conda-forge
lzo                       2.10              he774522_1000    conda-forge
mako                      1.2.4              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.3           py310h8d17308_0    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
mkl                       2022.1.0           h6a75c08_874    conda-forge
mpc                       1.3.1                h4ff82f8_0    conda-forge
mpfr                      4.2.0                h64bf75a_0    conda-forge
mpir                      3.0.0             he025d50_1002    conda-forge
numpy                     1.25.2          py310hd02465a_0    conda-forge
openjpeg                  2.5.0                ha2aaf27_2    conda-forge
openssl                   3.1.2                hcfcfb64_0    conda-forge
packaging                 23.1               pyhd8ed1ab_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
passlib                   1.7.4              pyh9f0ad1d_0    conda-forge
pcre2                     10.40                h17e33f8_0    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pip                       23.2.1             pyhd8ed1ab_0    conda-forge
pixman                    0.40.0               h8ffe710_0    conda-forge
pluggy                    1.3.0              pyhd8ed1ab_0    conda-forge
poppler                   23.08.0              h45d20d0_0    conda-forge
poppler-data              0.4.12               hd8ed1ab_0    conda-forge
postgresql                15.4                 hc80876b_0    conda-forge
proj                      9.2.1                h660b3b0_0    conda-forge
prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
psycopg2                  2.9.3           py310h709a2f1_2    conda-forge
pthreads-win32            2.9.1                hfa6e2cd_3    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pyasn1                    0.4.8                      py_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pydantic                  2.0.3              pyhd8ed1ab_1    conda-forge
pydantic-core             2.3.0           py310h87d50f1_0    conda-forge
pygments                  2.16.1             pyhd8ed1ab_0    conda-forge
pyopenssl                 23.2.0             pyhd8ed1ab_1    conda-forge
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyh0701188_6    conda-forge
pystac                    1.8.2              pyhd8ed1ab_0    conda-forge
pytest                    7.3.1              pyhd8ed1ab_0    conda-forge
python                    3.10.10         h4de0772_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-jose               3.3.0              pyh6c4a22f_1    conda-forge
python-multipart          0.0.6              pyhd8ed1ab_0    conda-forge
python_abi                3.10                    3_cp310    conda-forge
pyyaml                    6.0.1           py310h8d17308_0    conda-forge
rasterio                  1.3.8           py310h29ae91d_0    conda-forge
requests                  2.29.0             pyhd8ed1ab_0    conda-forge
rsa                       4.9                pyhd8ed1ab_0    conda-forge
s3transfer                0.6.2              pyhd8ed1ab_0    conda-forge
setuptools                68.1.2             pyhd8ed1ab_0    conda-forge
shapely                   2.0.1           py310h839b4a8_2    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.1.10               hfb803bf_0    conda-forge
sniffio                   1.3.0              pyhd8ed1ab_0    conda-forge
snuggs                    1.4.7                      py_0    conda-forge
sqlalchemy                2.0.12          py310h8d17308_0    conda-forge
sqlite                    3.43.0               hcfcfb64_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
starlette                 0.27.0             pyhd8ed1ab_0    conda-forge
tbb                       2021.10.0            h91493d7_0    conda-forge
tiledb                    2.16.3               h1ffc264_1    conda-forge
tk                        8.6.12               h8ffe710_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
traitlets                 5.9.0              pyhd8ed1ab_0    conda-forge
typing-extensions         4.7.1                hd8ed1ab_0    conda-forge
typing_extensions         4.7.1              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
ucrt                      10.0.22621.0         h57928b3_0    conda-forge
urllib3                   1.26.15            pyhd8ed1ab_0    conda-forge
uvicorn                   0.22.0          py310h5588dad_0    conda-forge
vc                        14.3                h64f974e_17    conda-forge
vc14_runtime              14.36.32532         hfdfe4a8_17    conda-forge
vs2015_runtime            14.36.32532         h05e6639_17    conda-forge
wcwidth                   0.2.6              pyhd8ed1ab_0    conda-forge
wheel                     0.41.2             pyhd8ed1ab_0    conda-forge
win_inet_pton             1.1.0              pyhd8ed1ab_6    conda-forge
xerces-c                  3.2.4                h63175ca_3    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
yaml                      0.2.5                h8ffe710_2    conda-forge
zipp                      3.16.2             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               hcfcfb64_5    conda-forge
zstd                      1.5.5                h12be248_0    conda-forge

Environment info

active environment : ddb-api
    active env location : C:\Users\nbrown\Miniconda3\envs\ddb-api
            shell level : 2
       user config file : C:\Users\nbrown\.condarc
 populated config files : C:\Users\nbrown\.condarc
          conda version : 23.9.0
    conda-build version : not installed
         python version : 3.8.17.final.0
       virtual packages : __archspec=1=x86_64
                          __win=0=0
       base environment : C:\Users\nbrown\Miniconda3  (writable)
      conda av data dir : C:\Users\nbrown\Miniconda3\etc\conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/win-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : C:\Users\nbrown\Miniconda3\pkgs
                          C:\Users\nbrown\.conda\pkgs
                          C:\Users\nbrown\AppData\Local\conda\conda\pkgs
       envs directories : C:\Users\nbrown\Miniconda3\envs
                          C:\Users\nbrown\.conda\envs
                          C:\Users\nbrown\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/23.9.0 requests/2.31.0 CPython/3.8.17 Windows/10 Windows/10.0.19045
          administrator : False
             netrc file : None
           offline mode : False
xylar commented 8 months ago

@NorahBrown, sorry you're having trouble. It would be surprising to me if such a basic operation in shapely were broken for all Windows users. I've added your test to our build in #131 so let's see if we can reproduce the issue here, too.

I'm not terribly expert at windows debugging but nothing obvious stands out to me in what you listed that might explain the issue you're having, unfortunately.

@conda-forge/shapely, anyone else have suggestions for what could be causing this?

xylar commented 8 months ago

@NorahBrown, the answer seems to be, no, we're not reproducing the problem in #131 so it seems somehow to be something more specific to how you have things installed. Hopefully, one of the other maintainers will have a suggestion for how to debug further.

akrherz commented 8 months ago

Shrug, I wonder why your environment has shapely 2.0.1, instead of the current 2.0.2 release?

ocefpaf commented 8 months ago

Do you have another installation of geos by any chance? That could cause a DLL hell and segfaults.

jorisvandenbossche commented 8 months ago

FWIW, for a correct installation of shapely 2.0.1, running shapely.speedups.disable() can never have any effect, because that function is a no-op: https://github.com/shapely/shapely/blob/2.0.1/shapely/speedups.py

If calling speedups has an effect, my guess is that you actually are somehow importing shapely 1.8 (or older). For those versions, disabling speedups can indeed fix installation issues if at runtime it is loading a different GEOS version than the one shapely was built against (because there is both cython code built against GEOS and ctypes code loading GEOS only at runtime).

jorisvandenbossche commented 8 months ago

So in your environment where you observe the issue, can you do:

import shapely
shapely.__version__
shapely.__path__
import shapely.geos
shapely.geos.lgeos.geos_version  # or shapely.geos.geos_version for more recent versions
NorahBrown commented 8 months ago

@jorisvandenbossche Thank you for the insight and guidance. You are correct. Even though I installed shapely 2.0 in my conda environment, shapely 1.8 was the version being loaded.
It appears that %USERPROFILE%\AppData\Roaming site-packages are prepended to the conda environment search path before the conda environments site-packages. In my case the %USERPROFILE%\AppData\Roaming\Python\Python310\site-packages has shapely 1.8 installed.

I understand that the issue is out of scope for shapely-feedstock. I humbly suggest that having the roaming site-packages added after the conda environment's site-packages would be more aligned with expectations for an isolated conda environment. For now I have temporarily renamed the roaming site-packages directory but a more permanent solution would be ideal.

%USERPROFILE%/AppData/Roaming/Python/Python310/site-packages exists

>>> import sys
>>> sys.path
[...,'C:\\Users\\nbrown\\Miniconda3\\envs\\ddb-api', 'C:\\Users\\nbrown\\AppData\\Roaming\\Python\\Python310\\site-packages', 'C:\\Users\\nbrown\\Miniconda3\\envs\\ddb-api\\lib\\site-packages']
>>> import shapely
>>> shapely.__version__
'1.8.4'
>>> import shapely.geos
>>> shapely.geos.lgeos.geos_version
(3, 8, 0)

%USERPROFILE%/AppData/Roaming/Python/Python310/site-packages does not exist

import sys
sys.path
[...,'C:\\Users\\nbrown\\Miniconda3\\envs\\ddb-api', 'C:\\Users\\nbrown\\Miniconda3\\envs\\ddb-api\\lib\\site-packages']
>>> import shapely
>>> shapely.__version__
'2.0.2'
>>> import shapely.geos
>>> shapely.geos.geos_version_string
'3.12.0-CAPI-1.18.0'