Closed gsakkis closed 3 years ago
Is the PDAL base library installed? It isn't included in the Python PDAL wheel (which is just the extension and PDAL filters.python
).
Yes it is (built from source):
pdal --version
-----------------------------------------------------------------------------------------------------------------------------------
pdal 2.2.0 (git-version: Release)
-----------------------------------------------------------------------------------------------------------------------------------
I managed to build pdal-python manually from source after installing scikit-build:
$ python setup.py build
Traceback (most recent call last):
File "setup.py", line 22, in <module>
from skbuild import setup
ModuleNotFoundError: No module named 'skbuild'
$ pip3 install scikit-build
Collecting scikit-build
Downloading scikit_build-0.11.1-py2.py3-none-any.whl (72 kB)
|████████████████████████████████| 72 kB 1.5 MB/s
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from scikit-build) (20.7)
Requirement already satisfied: wheel>=0.29.0 in /usr/lib/python3/dist-packages (from scikit-build) (0.34.2)
Requirement already satisfied: setuptools>=28.0.0; python_version >= "3" in /usr/lib/python3/dist-packages (from scikit-build) (45.2.0)
Requirement already satisfied: distro in /usr/lib/python3/dist-packages (from scikit-build) (1.4.0)
Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.8/dist-packages/pyparsing-3.0.0b1-py3.8.egg (from packaging->scikit-build) (3.0.0b1)
Installing collected packages: scikit-build
Successfully installed scikit-build-0.11.1
$ python setup.py build
--------------------------------------------------------------------------------
-- Trying "Ninja" generator
--------------------------------
---------------------------
----------------------
-----------------
------------
-------
--
Not searching for unused variables given on the command line.
-- The C compiler identification is GNU 9.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- The CXX compiler identification is GNU 9.3.0
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/data/archives/python-2.3.6/_cmake_test_compile/build
--
-------
------------
-----------------
----------------------
---------------------------
--------------------------------
-- Trying "Ninja" generator - success
--------------------------------------------------------------------------------
Configuring Project
Working directory:
/mnt/data/archives/python-2.3.6/_skbuild/linux-x86_64-3.8/cmake-build
Command:
cmake /mnt/data/archives/python-2.3.6 -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/mnt/data/archives/python-2.3.6/_skbuild/linux-x86_64-3.8/cmake-install -DPYTHON_EXECUTABLE:FILEPATH=/usr/bin/python -DPYTHON_VERSION_STRING:STRING=3.8.5 -DPYTHON_INCLUDE_DIR:PATH=/usr/include/python3.8 -DPYTHON_LIBRARY:FILEPATH=/usr/lib/x86_64-linux-gnu/libpython3.8.so -DSKBUILD:BOOL=TRUE -DCMAKE_MODULE_PATH:PATH=/usr/local/lib/python3.8/dist-packages/skbuild/resources/cmake -DCMAKE_BUILD_TYPE:STRING=Release
-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Python3: /usr/bin/python3.8 (found version "3.8.5") found components: Interpreter Development NumPy
-- Found PythonInterp: /usr/bin/python (found version "3.8.5")
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.8.so (found version "3.8.5")
-- Found Cython: /usr/bin/cython3
-- The project is built using scikit-build
-- Performing Test Weak Link MODULE -> SHARED (gnu_ld_ignore) - Success
_modinit_prefix:PyInit_
-- PROJECT_NAME: PDAL
-- PROJECT_NAME: PDAL
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/data/archives/python-2.3.6/_skbuild/linux-x86_64-3.8/cmake-build
[10/18] Building CXX object pdal/CMakeFiles/libpdalpython.dir/PyPipeline.cpp.o
In file included from /usr/include/python3.8/numpy/ndarraytypes.h:1830,
from /usr/include/python3.8/numpy/ndarrayobject.h:12,
from /usr/include/python3.8/numpy/arrayobject.h:4,
from ../../../pdal/PyPipeline.cpp:42:
/usr/include/python3.8/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
17 | #warning "Using deprecated NumPy API, disable it with " \
| ^~~~~~~
[11/18] Building CXX object pdal/CMakeFiles/libpdalpython.dir/libpdalpython.cxx.o
In file included from /usr/include/python3.8/numpy/ndarraytypes.h:1830,
from /usr/include/python3.8/numpy/ndarrayobject.h:12,
from /usr/include/python3.8/numpy/arrayobject.h:4,
from pdal/libpdalpython.cxx:619:
/usr/include/python3.8/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
17 | #warning "Using deprecated NumPy API, disable it with " \
| ^~~~~~~
[12/18] Building CXX object pdal/CMakeFiles/libpdalpython.dir/PyArray.cpp.o
In file included from /usr/include/python3.8/numpy/ndarraytypes.h:1830,
from ../../../pdal/PyArray.hpp:37,
from ../../../pdal/PyArray.cpp:35:
/usr/include/python3.8/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
17 | #warning "Using deprecated NumPy API, disable it with " \
| ^~~~~~~
[17/18] Install the project...
-- Install configuration: "Release"
-- Installing: /mnt/data/archives/python-2.3.6/_skbuild/linux-x86_64-3.8/cmake-install/pdal/libpdal_plugin_reader_numpy.so
-- Set runtime path of "/mnt/data/archives/python-2.3.6/_skbuild/linux-x86_64-3.8/cmake-install/pdal/libpdal_plugin_reader_numpy.so" to ""
-- Installing: /mnt/data/archives/python-2.3.6/_skbuild/linux-x86_64-3.8/cmake-install/pdal/libpdal_plugin_filter_python.so
-- Set runtime path of "/mnt/data/archives/python-2.3.6/_skbuild/linux-x86_64-3.8/cmake-install/pdal/libpdal_plugin_filter_python.so" to ""
-- Installing: /mnt/data/archives/python-2.3.6/_skbuild/linux-x86_64-3.8/cmake-install/pdal/libpdalpython.cpython-38-x86_64-linux-gnu.so
-- Set runtime path of "/mnt/data/archives/python-2.3.6/_skbuild/linux-x86_64-3.8/cmake-install/pdal/libpdalpython.cpython-38-x86_64-linux-gnu.so" to ""
copying pdal/array.py -> _skbuild/linux-x86_64-3.8/cmake-install/pdal/array.py
copying pdal/__init__.py -> _skbuild/linux-x86_64-3.8/cmake-install/pdal/__init__.py
copying pdal/dimension.py -> _skbuild/linux-x86_64-3.8/cmake-install/pdal/dimension.py
copying pdal/pio.py -> _skbuild/linux-x86_64-3.8/cmake-install/pdal/pio.py
copying pdal/pipeline.py -> _skbuild/linux-x86_64-3.8/cmake-install/pdal/pipeline.py
running build
running build_py
creating _skbuild/linux-x86_64-3.8/setuptools/lib
creating _skbuild/linux-x86_64-3.8/setuptools/lib/pdal
copying _skbuild/linux-x86_64-3.8/cmake-install/pdal/array.py -> _skbuild/linux-x86_64-3.8/setuptools/lib/pdal
copying _skbuild/linux-x86_64-3.8/cmake-install/pdal/__init__.py -> _skbuild/linux-x86_64-3.8/setuptools/lib/pdal
copying _skbuild/linux-x86_64-3.8/cmake-install/pdal/dimension.py -> _skbuild/linux-x86_64-3.8/setuptools/lib/pdal
copying _skbuild/linux-x86_64-3.8/cmake-install/pdal/pio.py -> _skbuild/linux-x86_64-3.8/setuptools/lib/pdal
copying _skbuild/linux-x86_64-3.8/cmake-install/pdal/pipeline.py -> _skbuild/linux-x86_64-3.8/setuptools/lib/pdal
copying _skbuild/linux-x86_64-3.8/cmake-install/pdal/libpdal_plugin_reader_numpy.so -> _skbuild/linux-x86_64-3.8/setuptools/lib/pdal
copying _skbuild/linux-x86_64-3.8/cmake-install/pdal/libpdal_plugin_filter_python.so -> _skbuild/linux-x86_64-3.8/setuptools/lib/pdal
copying _skbuild/linux-x86_64-3.8/cmake-install/pdal/libpdalpython.cpython-38-x86_64-linux-gnu.so -> _skbuild/linux-x86_64-3.8/setuptools/lib/pdal
copied 5 files
running build_ext
very strange that it is not installing and finding the cython shim. Are you using a virtual environment or conda or something? I'm stumped.
No, just the system python on a Ubuntu 20.04 vm. Btw I uninstalled it and tried to reinstall it via pip again in case the problem was the (previously) missing scikit-build but still the same, it still didn't build libpdalpython.
I ran pip install with --verbose, the only thing that stands out is skipping skbuild (no CMakeLists.txt found)
. Related logs:
Getting requirements to build wheel: started
Running command /usr/bin/python3 /tmp/tmpf2yl8qc2 get_requires_for_build_wheel /tmp/tmp4o_6bilv
skipping skbuild (no CMakeLists.txt found)
running egg_info
creating PDAL.egg-info
writing PDAL.egg-info/PKG-INFO
writing dependency_links to PDAL.egg-info/dependency_links.txt
writing top-level names to PDAL.egg-info/top_level.txt
writing manifest file 'PDAL.egg-info/SOURCES.txt'
reading manifest file 'PDAL.egg-info/SOURCES.txt'
writing manifest file 'PDAL.egg-info/SOURCES.txt'
Getting requirements to build wheel: finished with status 'done'
Created temporary directory: /tmp/pip-modern-metadata-mcmcoiw6
Preparing wheel metadata: started
Running command /usr/bin/python3 /tmp/tmpx84zw4pk prepare_metadata_for_build_wheel /tmp/tmp2smg593_
skipping skbuild (no CMakeLists.txt found)
running dist_info
creating /tmp/pip-modern-metadata-mcmcoiw6/PDAL.egg-info
writing /tmp/pip-modern-metadata-mcmcoiw6/PDAL.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-modern-metadata-mcmcoiw6/PDAL.egg-info/dependency_links.txt
writing top-level names to /tmp/pip-modern-metadata-mcmcoiw6/PDAL.egg-info/top_level.txt
writing manifest file '/tmp/pip-modern-metadata-mcmcoiw6/PDAL.egg-info/SOURCES.txt'
reading manifest file '/tmp/pip-modern-metadata-mcmcoiw6/PDAL.egg-info/SOURCES.txt'
writing manifest file '/tmp/pip-modern-metadata-mcmcoiw6/PDAL.egg-info/SOURCES.txt'
creating '/tmp/pip-modern-metadata-mcmcoiw6/PDAL.dist-info'
Preparing wheel metadata: finished with status 'done'
Source in /tmp/pip-install-dhw18kyh/pdal has version 2.3.6, which satisfies requirement pdal from https://files.pythonhosted.org/packages/1e/b3/1e736d42e0579048b1a4f9b686cd252a18d230603de2985220d9bc9cda4a/PDAL-2.3.6.tar.gz#sha256=b5e22c79043081e6b60d948af1f1036e4a5b1a503f843c0cb49cb669eff59c5e
Removed pdal from https://files.pythonhosted.org/packages/1e/b3/1e736d42e0579048b1a4f9b686cd252a18d230603de2985220d9bc9cda4a/PDAL-2.3.6.tar.gz#sha256=b5e22c79043081e6b60d948af1f1036e4a5b1a503f843c0cb49cb669eff59c5e from build tracker '/tmp/pip-req-tracker-kp1yzcm1'
uilding wheels for collected packages: pdal
Created temporary directory: /tmp/pip-wheel-8sve26bh
Destination directory: /tmp/pip-wheel-8sve26bh
Building wheel for pdal (PEP 517): started
Running command /usr/bin/python3 /tmp/tmpkeayjl3_ build_wheel /tmp/tmp1yominh8
skipping skbuild (no CMakeLists.txt found)
running bdist_wheel
running build
running build_py
creating _skbuild
creating _skbuild/linux-x86_64-3.8
creating _skbuild/linux-x86_64-3.8/setuptools
creating _skbuild/linux-x86_64-3.8/setuptools/lib
creating _skbuild/linux-x86_64-3.8/setuptools/lib/pdal
copying pdal/__init__.py -> _skbuild/linux-x86_64-3.8/setuptools/lib/pdal
copying pdal/array.py -> _skbuild/linux-x86_64-3.8/setuptools/lib/pdal
copying pdal/pio.py -> _skbuild/linux-x86_64-3.8/setuptools/lib/pdal
copying pdal/dimension.py -> _skbuild/linux-x86_64-3.8/setuptools/lib/pdal
copying pdal/pipeline.py -> _skbuild/linux-x86_64-3.8/setuptools/lib/pdal
copied 5 files
installing to _skbuild/linux-x86_64-3.8/setuptools/bdist.linux-x86_64/wheel
running install
running install_lib
creating _skbuild/linux-x86_64-3.8/setuptools/bdist.linux-x86_64
creating _skbuild/linux-x86_64-3.8/setuptools/bdist.linux-x86_64/wheel
creating _skbuild/linux-x86_64-3.8/setuptools/bdist.linux-x86_64/wheel/pdal
copying _skbuild/linux-x86_64-3.8/setuptools/lib/pdal/__init__.py -> _skbuild/linux-x86_64-3.8/setuptools/bdist.linux-x86_64/wheel/pdal
copying _skbuild/linux-x86_64-3.8/setuptools/lib/pdal/array.py -> _skbuild/linux-x86_64-3.8/setuptools/bdist.linux-x86_64/wheel/pdal
copying _skbuild/linux-x86_64-3.8/setuptools/lib/pdal/pio.py -> _skbuild/linux-x86_64-3.8/setuptools/bdist.linux-x86_64/wheel/pdal
copying _skbuild/linux-x86_64-3.8/setuptools/lib/pdal/dimension.py -> _skbuild/linux-x86_64-3.8/setuptools/bdist.linux-x86_64/wheel/pdal
copying _skbuild/linux-x86_64-3.8/setuptools/lib/pdal/pipeline.py -> _skbuild/linux-x86_64-3.8/setuptools/bdist.linux-x86_64/wheel/pdal
copied 5 files
running install_egg_info
running egg_info
writing PDAL.egg-info/PKG-INFO
writing dependency_links to PDAL.egg-info/dependency_links.txt
writing top-level names to PDAL.egg-info/top_level.txt
reading manifest file 'PDAL.egg-info/SOURCES.txt'
writing manifest file 'PDAL.egg-info/SOURCES.txt'
Copying PDAL.egg-info to _skbuild/linux-x86_64-3.8/setuptools/bdist.linux-x86_64/wheel/PDAL-2.3.6-py3.8.egg-info
running install_scripts
copied 0 files
creating _skbuild/linux-x86_64-3.8/setuptools/bdist.linux-x86_64/wheel/PDAL-2.3.6.dist-info/WHEEL
creating '/tmp/pip-wheel-8sve26bh/tmpgm2gmi6q/PDAL-2.3.6-py3-none-any.whl' and adding '_skbuild/linux-x86_64-3.8/setuptools/bdist.linux-x86_64/wheel' to it
adding 'pdal/__init__.py'
adding 'pdal/array.py'
adding 'pdal/dimension.py'
adding 'pdal/pio.py'
adding 'pdal/pipeline.py'
adding 'PDAL-2.3.6.dist-info/METADATA'
adding 'PDAL-2.3.6.dist-info/WHEEL'
adding 'PDAL-2.3.6.dist-info/top_level.txt'
adding 'PDAL-2.3.6.dist-info/RECORD'
removing _skbuild/linux-x86_64-3.8/setuptools/bdist.linux-x86_64/wheel
Building wheel for pdal (PEP 517): finished with status 'done'
Created wheel for pdal: filename=PDAL-2.3.6-py3-none-any.whl size=6467 sha256=d9f9498e669ae2216edeb180246e1493d77b5a9a97d5c8bb1b7610621696e7db
Stored in directory: /tmp/pip-ephem-wheel-cache-_qpcxnxv/wheels/12/ef/4f/30f6a1126d2a413b49a6208034bf1262b219b848ebd3025ebd
Successfully built pdal
Installing collected packages: pdal
Created temporary directory: /tmp/pip-unpacked-wheel-3ks24sj1
Successfully installed pdal-2.3.6
Cleaning up...
Removing source in /tmp/pip-install-dhw18kyh/pdal
Removed build tracker: '/tmp/pip-req-tracker-kp1yzcm1'
Strange. If _skbuild isn't being built, the cython shim isn't being compiled. Please compare to our CI setup and look if you see any interesting discrepancies https://github.com/PDAL/python/blob/master/.github/workflows/build.yml
@gsakkis I'm still getting the same issue you described after trying to install pdal using pip. Is this expected to be corrected on the next release with the PR you just merged? Or should I follow any other steps in order to get it running? Any help will be more than welcome :) Thanks!
I'll make a new release today.
superb! thanks! :)
@hobu I just tested it and it worked! thanks a lot :)
Installing via pip seems to succeed but the installation is incomplete: