search

conda-forge / qgis-feedstock

A conda-smithy repository for qgis.
BSD 3-Clause "New" or "Revised" License
41 stars 26 forks source link

Running Fix Geometries using PyQGIS outside QGIS does not perform as expected #436

Open EricSamson-Tract opened 4 weeks ago

EricSamson-Tract commented 4 weeks ago

Solution to issue cannot be found in the documentation.

Issue

When running fix geometries from a conda environment with QGIS installed, fix geometries will work on all geometries except for 1 or 2. When running fix geometries within QGIS, it will fix all geometries (based on geos).

`def zonal_histogram_qgis(gdf, raster_path): temp_dir = tempfile.mkdtemp() temp_shapefile = os.path.join(temp_dir, 'parcels.shp') gdf.to_file(temp_shapefile)

input_layer = QgsVectorLayer(temp_shapefile, 'parcels', 'ogr')

fixed_shapefile = os.path.join(temp_dir, 'parcels_fixed.shp')
fix_params = {
    'INPUT': input_layer,
    'METHOD': 1,
    'OUTPUT': fixed_shapefile
}
Processing.runAlgorithm("native:fixgeometries", fix_params)

out_shape = os.path.join(temp_dir, 'zonal_stats.shp')
zone_params = {
    'INPUT_RASTER': raster_path,
    'RASTER_BAND': 1,
    'INPUT_VECTOR': fixed_shapefile2,
    'COLUMN_PREFIX': 'value_',
    'OUTPUT': out_shape
}
feedback = QgsProcessingFeedback()
Processing.runAlgorithm("native:zonalhistogram", zone_params, feedback=feedback)

zonal_gdf = gpd.read_file(out_shape)

return zonal_gdf`

I unfortunately can't attach data. I am running this on a dataset of around 30,000 polygons, derived from parcel data. Running something similar from within the python window in QGIS works as expected, the gui tools as well. I am not sure what is occurring here.

Installed packages

# Name                    Version                   Build  Channel
affine                    2.4.0                    pypi_0    pypi
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
attrs                     23.2.0                   pypi_0    pypi
aws-c-auth                0.7.20               h6823eb1_0    conda-forge
aws-c-cal                 0.6.12               hc83774a_0    conda-forge
aws-c-common              0.9.17               h2466b09_0    conda-forge
aws-c-compression         0.2.18               hc83774a_4    conda-forge
aws-c-event-stream        0.4.2               hc6c0aac_10    conda-forge
aws-c-http                0.8.1               hced5053_13    conda-forge
aws-c-io                  0.14.8               hebaacdb_0    conda-forge
aws-c-mqtt                0.10.4               hdafd9a4_2    conda-forge
aws-c-s3                  0.5.8                h7a83f0e_3    conda-forge
aws-c-sdkutils            0.1.16               hc83774a_0    conda-forge
aws-checksums             0.1.18               hc83774a_4    conda-forge
aws-crt-cpp               0.26.8              h4438f58_10    conda-forge
aws-sdk-cpp               1.11.329             h12f3f85_0    conda-forge
azure-core-cpp            1.11.1               h249a519_1    conda-forge
azure-identity-cpp        1.6.0                h91493d7_1    conda-forge
azure-storage-blobs-cpp   12.10.0              h91493d7_1    conda-forge
azure-storage-common-cpp  12.5.0               h91493d7_4    conda-forge
blas                      2.122                       mkl    conda-forge
blas-devel                3.9.0              22_win64_mkl    conda-forge
blosc                     1.21.5               hbd69f2e_1    conda-forge
brotli-python             1.1.0           py312h53d5487_1    conda-forge
bzip2                     1.0.8                hcfcfb64_5    conda-forge
c-ares                    1.28.1               hcfcfb64_0    conda-forge
ca-certificates           2024.2.2             h56e8100_0    conda-forge
cairo                     1.18.0               h1fef639_0    conda-forge
ceres-solver              2.2.0                h0d88682_3    conda-forge
certifi                   2024.2.2           pyhd8ed1ab_0    conda-forge
cfitsio                   4.4.0                h9b0cee5_1    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
click                     8.1.7                    pypi_0    pypi
click-plugins             1.1.1                    pypi_0    pypi
cligj                     0.7.2                    pypi_0    pypi
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
comm                      0.2.2              pyhd8ed1ab_0    conda-forge
coverage                  7.5.1           py312h4389bb4_0    conda-forge
curl                      8.7.1                hd5e4a3a_0    conda-forge
dataclasses               0.8                pyhc8e2a94_3    conda-forge
debugpy                   1.8.1           py312h53d5487_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
draco                     1.5.7                h181d51b_0    conda-forge
eigen                     3.4.0                h91493d7_0    conda-forge
et-xmlfile                1.1.0                    pypi_0    pypi
exceptiongroup            1.2.0              pyhd8ed1ab_2    conda-forge
executing                 2.0.1              pyhd8ed1ab_0    conda-forge
exiv2                     0.28.2               hadc2d18_0    conda-forge
expat                     2.6.2                h63175ca_0    conda-forge
fiona                     1.9.6                    pypi_0    pypi
fmt                       10.2.1               h181d51b_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 h77eed37_2    conda-forge
fontconfig                2.14.2               hbde0cde_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
freetype                  2.12.1               hdaf720e_2    conda-forge
freexl                    2.0.0                h8276f4a_0    conda-forge
future                    1.0.0              pyhd8ed1ab_0    conda-forge
gdal                      3.8.5           py312hea5013e_4    conda-forge
geopandas                 0.14.4                   pypi_0    pypi
geos                      3.12.1               h1537add_0    conda-forge
geotiff                   1.7.1               hfbd0779_17    conda-forge
gflags                    2.2.2             ha925a31_1004    conda-forge
glib                      2.80.2               h0df6a38_0    conda-forge
glib-tools                2.80.2               h2f9d560_0    conda-forge
glog                      0.7.0                h9cd36e5_0    conda-forge
greenlet                  3.0.3                    pypi_0    pypi
gsl                       2.7                  hdfb1a43_0    conda-forge
gst-plugins-base          1.24.3               hba88be7_0    conda-forge
gstreamer                 1.24.3               h5006eae_0    conda-forge
hdf4                      4.2.15               h5557f11_7    conda-forge
hdf5                      1.14.3          nompi_h73e8ff5_101    conda-forge
httplib2                  0.22.0             pyhd8ed1ab_0    conda-forge
icu                       73.2                 h63175ca_0    conda-forge
idna                      3.7                pyhd8ed1ab_0    conda-forge
importlib-metadata        7.1.0              pyha770c72_0    conda-forge
importlib_metadata        7.1.0                hd8ed1ab_0    conda-forge
intel-openmp              2024.1.0           h57928b3_965    conda-forge
ipykernel                 6.29.3             pyha63f2e9_0    conda-forge
ipython                   8.24.0             pyh7428d3b_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
jupyter_client            8.6.1              pyhd8ed1ab_0    conda-forge
jupyter_core              5.7.2           py312h2e8e312_0    conda-forge
kealib                    1.5.3                h6c43f9b_1    conda-forge
khronos-opencl-icd-loader 2023.04.17           h64bf75a_0    conda-forge
krb5                      1.21.2               heb0366b_0    conda-forge
laz-perf                  3.4.0                h91493d7_0    conda-forge
lcms2                     2.16                 h67d730c_0    conda-forge
lerc                      4.0.0                h63175ca_0    conda-forge
libabseil                 20240116.2      cxx17_h63175ca_0    conda-forge
libaec                    1.1.3                h63175ca_0    conda-forge
libarchive                3.7.2                h313118b_1    conda-forge
libblas                   3.9.0              22_win64_mkl    conda-forge
libboost-headers          1.85.0               h57928b3_1    conda-forge
libbrotlicommon           1.1.0                hcfcfb64_1    conda-forge
libbrotlidec              1.1.0                hcfcfb64_1    conda-forge
libbrotlienc              1.1.0                hcfcfb64_1    conda-forge
libcblas                  3.9.0              22_win64_mkl    conda-forge
libclang13                18.1.5          default_hf64faad_0    conda-forge
libcrc32c                 1.1.2                h0e60522_0    conda-forge
libcurl                   8.7.1                hd5e4a3a_0    conda-forge
libdeflate                1.20                 hcfcfb64_0    conda-forge
libexpat                  2.6.2                h63175ca_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libgdal                   3.8.5                hfb9f81c_4    conda-forge
libglib                   2.80.2               h0df6a38_0    conda-forge
libgoogle-cloud           2.24.0               h5e7cea3_0    conda-forge
libgoogle-cloud-storage   2.24.0               hce61461_0    conda-forge
libgrpc                   1.62.2               h5273850_0    conda-forge
libhwloc                  2.10.0          default_h8125262_1001    conda-forge
libiconv                  1.17                 hcfcfb64_2    conda-forge
libintl                   0.22.5               h5728263_2    conda-forge
libintl-devel             0.22.5               h5728263_2    conda-forge
libjpeg-turbo             3.0.0                hcfcfb64_1    conda-forge
libkml                    1.3.0             haf3e7a6_1018    conda-forge
liblapack                 3.9.0              22_win64_mkl    conda-forge
liblapacke                3.9.0              22_win64_mkl    conda-forge
libnetcdf                 4.9.2           nompi_h07c049d_113    conda-forge
libogg                    1.3.4                h8ffe710_1    conda-forge
libpng                    1.6.43               h19919ed_0    conda-forge
libpq                     16.3                 hab9416b_0    conda-forge
libprotobuf               4.25.3               h503648d_0    conda-forge
libre2-11                 2023.09.01           hf8d8778_2    conda-forge
librttopo                 1.1.0               h94c4f80_15    conda-forge
libsodium                 1.0.18               h8d14728_1    conda-forge
libspatialindex           1.9.3                h5a68840_5    conda-forge
libspatialite             5.1.0                h1b08085_6    conda-forge
libsqlite                 3.45.3               hcfcfb64_0    conda-forge
libssh2                   1.11.0               h7dfc565_0    conda-forge
libtiff                   4.6.0                hddb2be6_3    conda-forge
libvorbis                 1.3.7                h0e60522_0    conda-forge
libwebp                   1.4.0                h2466b09_0    conda-forge
libwebp-base              1.4.0                hcfcfb64_0    conda-forge
libxml2                   2.12.7               h283a6d9_0    conda-forge
libxslt                   1.1.39               h3df6e99_0    conda-forge
libzip                    1.10.1               h1d365fa_3    conda-forge
libzlib                   1.2.13               hcfcfb64_5    conda-forge
lxml                      5.2.2           py312h56c7e3b_0    conda-forge
lz4-c                     1.9.4                hcfcfb64_0    conda-forge
lzo                       2.10              hcfcfb64_1001    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
markupsafe                2.1.5           py312he70551f_0    conda-forge
matplotlib-inline         0.1.7              pyhd8ed1ab_0    conda-forge
minizip                   4.0.5                h5bed578_0    conda-forge
mkl                       2024.1.0           h66d3029_692    conda-forge
mkl-devel                 2024.1.0           h57928b3_692    conda-forge
mkl-include               2024.1.0           h66d3029_692    conda-forge
mock                      5.1.0              pyhd8ed1ab_0    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
nest-asyncio              1.6.0              pyhd8ed1ab_0    conda-forge
nitro                     2.7.dev8             h1537add_0    conda-forge
nose2                     0.9.2                      py_0    conda-forge
numpy                     1.26.4          py312h8753938_0    conda-forge
openjpeg                  2.5.2                h3d672ee_0    conda-forge
openpyxl                  3.1.2                    pypi_0    pypi
openssl                   3.3.0                h2466b09_2    conda-forge
owslib                    0.31.0             pyhd8ed1ab_0    conda-forge
packaging                 24.0               pyhd8ed1ab_0    conda-forge
pandas                    2.2.2                    pypi_0    pypi
parso                     0.8.4              pyhd8ed1ab_0    conda-forge
pcre2                     10.43                h17e33f8_0    conda-forge
pdal                      2.7.1                h78909d1_6    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pip                       24.0               pyhd8ed1ab_0    conda-forge
pixman                    0.43.4               h63175ca_0    conda-forge
platformdirs              4.2.2              pyhd8ed1ab_0    conda-forge
plotly                    5.22.0             pyhd8ed1ab_0    conda-forge
ply                       3.11               pyhd8ed1ab_2    conda-forge
poppler                   24.04.0              h747fd5a_0    conda-forge
poppler-data              0.4.12               hd8ed1ab_0    conda-forge
postgresql                16.3                 h7f155c9_0    conda-forge
proj                      9.4.0                he13c7e8_1    conda-forge
prompt-toolkit            3.0.42             pyha770c72_0    conda-forge
psutil                    5.9.8           py312he70551f_0    conda-forge
psycopg2                  2.9.9           py312hf50bb3c_0    conda-forge
pthreads-win32            2.9.1                hfa6e2cd_3    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pyarrow                   16.1.0                   pypi_0    pypi
pygments                  2.18.0             pyhd8ed1ab_0    conda-forge
pyparsing                 3.1.2              pyhd8ed1ab_0    conda-forge
pyproj                    3.6.1           py312h616b599_6    conda-forge
pyqt                      5.15.9          py312he09f080_5    conda-forge
pyqt5-sip                 12.12.2         py312h53d5487_5    conda-forge
pyqtwebkit                5.15.9          py312hca0710b_2    conda-forge
pysocks                   1.7.1              pyh0701188_6    conda-forge
python                    3.12.3          h2628c8c_0_cpython    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python_abi                3.12                    4_cp312    conda-forge
pytz                      2024.1             pyhd8ed1ab_0    conda-forge
pywin32                   306             py312h53d5487_2    conda-forge
pyyaml                    6.0.1           py312he70551f_1    conda-forge
pyzmq                     26.0.3          py312hd7027bb_0    conda-forge
qca                       2.3.8                h2624d1c_0    conda-forge
qgis                      3.36.3          py312hdab107f_0    conda-forge
qjson                     0.9.0             h04a78d6_1009    conda-forge
qscintilla2               2.14.1          py312hca0710b_0    conda-forge
qt-main                   5.15.8              hcef0176_21    conda-forge
qtkeychain                0.14.3               hf9d22a5_0    conda-forge
qtwebkit                  5.212               h4d8ddc9_16    conda-forge
qwt                       6.3.0                h9417a65_0    conda-forge
rasterio                  1.3.10                   pypi_0    pypi
re2                       2023.09.01           hd3b24a8_2    conda-forge
requests                  2.32.1             pyhd8ed1ab_0    conda-forge
setuptools                69.5.1             pyhd8ed1ab_0    conda-forge
shapely                   2.0.4                    pypi_0    pypi
sip                       6.7.12          py312h53d5487_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.2.0                hfb803bf_1    conda-forge
snuggs                    1.4.7                    pypi_0    pypi
spdlog                    1.13.0               h64d2f7d_0    conda-forge
sqlalchemy                2.0.30                   pypi_0    pypi
sqlite                    3.45.3               hcfcfb64_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
tbb                       2021.12.0            hc790b64_1    conda-forge
tenacity                  8.3.0              pyhd8ed1ab_0    conda-forge
tiledb                    2.22.0               h13740f5_4    conda-forge
tk                        8.6.13               h5226925_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tornado                   6.4             py312he70551f_0    conda-forge
traitlets                 5.14.3             pyhd8ed1ab_0    conda-forge
typing_extensions         4.11.0             pyha770c72_0    conda-forge
tzdata                    2024.1                   pypi_0    pypi
ucrt                      10.0.22621.0         h57928b3_0    conda-forge
uriparser                 0.9.8                h5a68840_0    conda-forge
urllib3                   2.2.1              pyhd8ed1ab_0    conda-forge
vc                        14.3                ha32ba9b_20    conda-forge
vc14_runtime              14.38.33135         h835141b_20    conda-forge
vs2015_runtime            14.38.33135         h22015db_20    conda-forge
wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
wheel                     0.43.0             pyhd8ed1ab_1    conda-forge
win_inet_pton             1.1.0              pyhd8ed1ab_6    conda-forge
xerces-c                  3.2.5                h63175ca_0    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
yaml                      0.2.5                h8ffe710_2    conda-forge
zeromq                    4.3.5                he1f189c_4    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               hcfcfb64_5    conda-forge
zstd                      1.5.6                h0ea2cb4_0    conda-forge

Environment info

active environment : qgis_dev
    active env location : \anaconda3\envs\qgis_dev
            shell level : 2
       user config file : \.condarc
 populated config files : 
          conda version : 23.10.0
    conda-build version : 3.25.0
         python version : 3.11.5.final.0
       virtual packages : __archspec=1=x86_64
                          __cuda=12.3=0
                          __win=0=0
       base environment : \anaconda3  (writable)
      conda av data dir : \anaconda3\etc\conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : \anaconda3\pkgs
                          \.conda\pkgs
                          \AppData\Local\conda\conda\pkgs
       envs directories : \anaconda3\envs
                          \.conda\envs
                          \AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/23.10.0 requests/2.31.0 CPython/3.11.5 Windows/10 Windows/10.0.22621 solver/libmamba conda-libmamba-solver/23.11.0 libmambapy/1.5.3 aau/0.4.3 c/8-vPlzrjxjaYu5zf2_lXHw s/RA-SHHiitLZsFiSdZ6iN2g e/U2nIOdCW9NhLdpQpCI0EzQ
          administrator : False
             netrc file : None
           offline mode : False
gillins commented 4 weeks ago

I see that you've installed rasterio via pypi which is known to cause conflicts with conda (they package their own GDAL library and is not recommended for production - see https://rasterio.readthedocs.io/en/stable/installation.html). Can you try again in a brand new environment with just qgis? And not install anything from pypi into it...