search

GeoNode / geonode-importer

MIT License
2 stars 15 forks source link

Unable to install geonode on python 3.11 due to gdal forced version #218

Closed miceno closed 7 months ago

miceno commented 8 months ago

GeoNode version: master (sha:a42f9f2426de444d51ab12046452893679d5803f) Installation type (vanilla, geonode-project): vanilla Installation method (manual, docker): manual Platform: Almalinux 9.3

I was following the instructions to setup geonode, but using python 3.11.

I used gdal36 library, so I installed it manually:

(geonode-4-master-py311) [geonode@server geonode-src]$ pip install gdal==3.6.4
Collecting gdal==3.6.4
  Using cached GDAL-3.6.4-cp311-cp311-linux_x86_64.whl
Installing collected packages: gdal
Successfully installed gdal-3.6.4

(geonode-4-master-py311) [geonode@server geonode-src]$ pip install pygdal=="`/usr/gdal36/bin/gdal-config --version`.*"
Collecting pygdal==3.6.4.*
  Using cached pygdal-3.6.4.11-cp311-cp311-linux_x86_64.whl
Collecting numpy>=1.0.0 (from pygdal==3.6.4.*)
  Obtaining dependency information for numpy>=1.0.0 from https://files.pythonhosted.org/packages/b6/ab/5b893944b1602a366893559bfb227fdfb3ad7c7629b2a80d039bb5924367/numpy-1.26.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Using cached numpy-1.26.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
Using cached numpy-1.26.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
Installing collected packages: numpy, pygdal
Successfully installed numpy-1.26.2 pygdal-3.6.4.11

While running command pip install -e ., a conflict raised because geonode-importer is installing gdal==3.2.2.1, but the virtual environment has gdal36. Here it is the output of the installation:


(geonode-4-master-py311) [geonode@server geonode-src]$ pip install -e . --upgrade
Obtaining file:///home/geonode/geonode-src
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting geonode-importer
  Obtaining dependency information for geonode-importer from https://files.pythonhosted.org/packages/e8/28/424ed3a099f9b79f4f48c8ead0b0c7b6d5c9eac74a0496db4b6291276f40/geonode_importer-1.0.5-py3-none-any.whl.metadata
  Using cached geonode_importer-1.0.5-py3-none-any.whl.metadata (752 bytes)
Requirement already satisfied: setuptools>=59 in /home/geonode/venv/geonode-4-master-py311/lib/python3.11/site-packages (from geonode-importer) (68.2.2)
**Collecting gdal==3.2.2.1 (from geonode-importer)
  Using cached GDAL-3.2.2.1.tar.gz (600 kB)**
  Preparing metadata (setup.py) ... done
Collecting pdok-geopackage-validator==0.8.0 (from geonode-importer)

...

Building wheels for collected packages: gdal, GeoNode
  Building wheel for gdal (setup.py) ... |                                                                                           error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [268 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-311
      creating build/lib.linux-x86_64-cpython-311/osgeo
      copying osgeo/__init__.py -> build/lib.linux-x86_64-cpython-311/osgeo
      copying osgeo/gdal.py -> build/lib.linux-x86_64-cpython-311/osgeo
      copying osgeo/gdal_array.py -> build/lib.linux-x86_64-cpython-311/osgeo
      copying osgeo/gdalconst.py -> build/lib.linux-x86_64-cpython-311/osgeo
      copying osgeo/gdalnumeric.py -> build/lib.linux-x86_64-cpython-311/osgeo
      copying osgeo/gnm.py -> build/lib.linux-x86_64-cpython-311/osgeo
      copying osgeo/ogr.py -> build/lib.linux-x86_64-cpython-311/osgeo
      copying osgeo/osr.py -> build/lib.linux-x86_64-cpython-311/osgeo
      creating build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/__init__.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/epsg_tr.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/esri2wkt.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gcps2vec.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gcps2wld.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gdal2xyz.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gdal_auth.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gdal_calc.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gdal_edit.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gdal_fillnodata.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gdal_merge.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gdal_pansharpen.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gdal_polygonize.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gdal_proximity.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gdal_retile.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gdal_sieve.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gdalchksum.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gdalcompare.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gdalident.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gdalimport.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/gdalmove.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/mkgraticule.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/ogrmerge.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/pct2rgb.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      copying osgeo/utils/rgb2pct.py -> build/lib.linux-x86_64-cpython-311/osgeo/utils
      running build_ext
...
      building 'osgeo._gdal_array' extension
      gcc -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format
-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchrono
us-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe 
-Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I../../port -I../../gcore -I../../alg -I../../ogr/ -I../../ogr/ogrsf_frmts -I../../gnm -I../../apps -I/home/geonode/venv/geonode-4-master-py311/include -I/usr/include/python3.11 -I/home/geonode/venv/geonode-4-master-py311/lib64/python3.11/site-packages/numpy/core/include -I/usr/gdal36/include -c extensions/gdal_array_wrap.cpp -o build/temp.linux-x86_64-cpython-311/extensions/gdal_array_wrap.o -I/usr/gdal36/include
      extensions/gdal_array_wrap.cpp:3584:25: error: ‘virtual const char* NUMPYDataset::_GetProjectionRef()’ marked ‘override’, but does not override
       3584 |     virtual const char *_GetProjectionRef(void) override;
            |                         ^~~~~~~~~~~~~~~~~
      extensions/gdal_array_wrap.cpp:3588:20: error: ‘virtual CPLErr NUMPYDataset::_SetProjection(const char*)’ marked ‘override’, but does not override
       3588 |     virtual CPLErr _SetProjection( const char * ) override;
            |                    ^~~~~~~~~~~~~~
      extensions/gdal_array_wrap.cpp:3597:25: error: ‘virtual const char* NUMPYDataset::_GetGCPProjection()’ marked ‘override’, but does not override
       3597 |     virtual const char *_GetGCPProjection() override;
            |                         ^~~~~~~~~~~~~~~~~
      extensions/gdal_array_wrap.cpp:3602:20: error: ‘virtual CPLErr NUMPYDataset::_SetGCPs(int, const GDAL_GCP*, const char*)’ marked ‘override’, but does not override
       3602 |     virtual CPLErr _SetGCPs( int nGCPCount, const GDAL_GCP *pasGCPList,
            |                    ^~~~~~~~
      extensions/gdal_array_wrap.cpp: In member function ‘virtual const OGRSpatialReference* NUMPYDataset::GetSpatialRef() const’:
      extensions/gdal_array_wrap.cpp:3586:16: error: ‘GetSpatialRefFromOldGetProjectionRef’ was not declared in this scope
       3586 |         return GetSpatialRefFromOldGetProjectionRef();
            |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      extensions/gdal_array_wrap.cpp: In member function ‘virtual CPLErr NUMPYDataset::SetSpatialRef(const OGRSpatialReference*)’:
      extensions/gdal_array_wrap.cpp:3590:16: error: ‘OldSetProjectionFromSetSpatialRef’ was not declared in this scope
       3590 |         return OldSetProjectionFromSetSpatialRef(poSRS);
            |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      extensions/gdal_array_wrap.cpp: In member function ‘virtual const OGRSpatialReference* NUMPYDataset::GetGCPSpatialRef() const’:
      extensions/gdal_array_wrap.cpp:3599:16: error: ‘GetGCPSpatialRefFromOldGetGCPProjection’ was not declared in this scope
       3599 |         return GetGCPSpatialRefFromOldGetGCPProjection();
            |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      extensions/gdal_array_wrap.cpp: In member function ‘virtual CPLErr NUMPYDataset::SetGCPs(int, const GDAL_GCP*, const OGRSpatialReference*)’:
      extensions/gdal_array_wrap.cpp:3607:16: error: ‘OldSetGCPsFromNew’ was not declared in this scope
       3607 |         return OldSetGCPsFromNew(nGCPCount, pasGCPList, poSRS);
            |                ^~~~~~~~~~~~~~~~~
      In file included from /usr/include/python3.11/Python.h:95,
                       from extensions/gdal_array_wrap.cpp:174:
      extensions/gdal_array_wrap.cpp: In function ‘int PyProgressProxy(double, const char*, void*)’:
      /usr/include/python3.11/ceval.h:34:34: warning: ‘PyObject* PyEval_CallObjectWithKeywords(PyObject*, PyObject*, PyObject*)’ is deprecated [-Wdeprecated-declarations]
         34 |     PyEval_CallObjectWithKeywords(callable, arg, (PyObject *)NULL)
            |     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      extensions/gdal_array_wrap.cpp:4279:16: note: in expansion of macro ‘PyEval_CallObject’
       4279 |     psResult = PyEval_CallObject( psInfo->psPyCallback, psArgs);
            |                ^~~~~~~~~~~~~~~~~
      /usr/include/python3.11/ceval.h:27:43: note: declared here
         27 | Py_DEPRECATED(3.9) PyAPI_FUNC(PyObject *) PyEval_CallObjectWithKeywords(
            |                                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      extensions/gdal_array_wrap.cpp: In function ‘PyObject* PyInit__gdal_array()’:
      extensions/gdal_array_wrap.cpp:1067:64: warning: ‘void PyEval_InitThreads()’ is deprecated [-Wdeprecated-declarations]
       1067 | #     define SWIG_PYTHON_INITIALIZE_THREADS  PyEval_InitThreads()
            |                                              ~~~~~~~~~~~~~~~~~~^~
      extensions/gdal_array_wrap.cpp:6935:3: note: in expansion of macro ‘SWIG_PYTHON_INITIALIZE_THREADS’
       6935 |   SWIG_PYTHON_INITIALIZE_THREADS;
            |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      In file included from /usr/include/python3.11/Python.h:95,
                       from extensions/gdal_array_wrap.cpp:174:
      /usr/include/python3.11/ceval.h:132:37: note: declared here
        132 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
            |                                     ^~~~~~~~~~~~~~~~~~
      g++ -shared -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g build/temp.linux-x86_64-cpython-311/extensions/gnm_wrap.o -L../../.libs -L../../ -L/usr/lib64 -L/usr/gdal36/lib -lgdal -o build/lib.linux-x86_64-cpython-311/osgeo/_gnm.cpython-311-x86_64-linux-gnu.so
      g++ -shared -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g build/temp.linux-x86_64-cpython-311/extensions/osr_wrap.o -L../../.libs -L../../ -L/usr/lib64 -L/usr/gdal36/lib -lgdal -o build/lib.linux-x86_64-cpython-311/osgeo/_osr.cpython-311-x86_64-linux-gnu.so
      g++ -shared -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g build/temp.linux-x86_64-cpython-311/extensions/ogr_wrap.o -L../../.libs -L../../ -L/usr/lib64 -L/usr/gdal36/lib -lgdal -o build/lib.linux-x86_64-cpython-311/osgeo/_ogr.cpython-311-x86_64-linux-gnu.so
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for gdal
  Running setup.py clean for gdal
  Building editable for GeoNode (pyproject.toml) ... done
  Created wheel for GeoNode: filename=GeoNode-4.2.0.dev0-0.editable-py2.py3-none-any.whl size=10133 sha256=29faa2a02921c2e9bc18e04631a23bfa161eb160c616d918e053b3f529fbad6f
  Stored in directory: /tmp/pip-ephem-wheel-cache-mqq_4ee8/wheels/68/78/77/cc9dcd7147f1143f06e115e772bdaf35077680472241e674c5
Successfully built GeoNode
Failed to build gdal
ERROR: Could not build wheels for gdal, which is required to install pyproject.toml-based projects

The expected behaviour was that the available version of gdal was valid.

miceno commented 8 months ago

Patching the requirements of geonode-importer allowed installing succesfully. Here it is the patch:

diff --git a/setup.py b/setup.py
index 213bae1..6f23cdf 100644
--- a/setup.py
+++ b/setup.py
@@ -33,7 +33,7 @@ setup(
     include_package_data=True,
     install_requires=[
         "setuptools>=59",
-        "gdal==3.2.2.1",
+        "gdal>=3.5",
         "pdok-geopackage-validator==0.8.0",
         "geonode-django-dynamic-model==0.4.0",
     ],

It also required some tuning of environment variables:

export PATH=$PATH:/usr/pgsql-13/bin:/usr/gdal36/bin
miceno commented 8 months ago

It is related to https://github.com/GeoNode/geonode/issues/11683

mattiagiupponi commented 8 months ago

Hi @miceno Thanks for opening the issue. The fixed version of the gdal is needed due a dependency for the gpkg validation library . We open in past an issue related to this on the dependency repo but it looks like was needed.

Since newer release of the library are published, we can try to upgrade the dependency to the lastest version and see if we can be less strict on that @giohappy

mattiagiupponi commented 8 months ago

@miceno Can you please try again using this branch? With this branch contains the library upgrade and i removed the fixed dependency of gdal

You can install the branch with:

pip install -e git+https://github.com/geosolutions-it/geonode-importer.git@ISSUE_218#egg=geonode_importer