Can't install python pdal neither for deployment in production nor in a venv environment

[shayannikoohemat]

I am trying to create a pypi package of my code with pdal dependency using bitbucket-pipeline and requirements.txt. I get the error similar to this issue: issue66. I also get the same error if I try to install python pdal in a venv enviroment on ubuntu 20.04 with python3.9. I only can make it work using conda installation conda create --yes --name pdal-env --channel conda-forge pdal python-pdal python=3.9 but I don't want to use conda for deployment of my code.

Here is the steps:

pipelines:
  branches:
    master:
      - step:
          name: Build and test
          image: python:3.9
          script:
            - apt-get update
            - apt-get install -y pdal
            - pip3 install --upgrade pip
            - pip3 install --no-cache-dir --index-url https://bitbucket:${NEXUS_PASSWORD}@nexus/repository/pypi/simple -r requirements.txt
            - make test
       - step:
       - ...

And my pyproject.toml file:

[build-system]
requires = [
    "scikit-build",
    "cmake",
    "ninja",
    "numpy",
    "pybind11[global]",
    "setuptools>=42",
    "wheel"
]
build-backend = "setuptools.build_meta"

requirement.txt file:

# Requirements for Python 3.9

laspy[lazrs]
numpy
pdal

and the ERROR I get:

Building wheels for collected packages: carrier
  Building wheel for carrier (pyproject.toml) ... done
  Created wheel for carrier: filename=carrier-0.1.BUILD_ID-py3-none-any.whl size=13800 
  Stored in directory: /foo/.cache/pip/wheels/85/12/71/7cda
  WARNING: Built wheel for carrier is invalid: Metadata 1.2 mandates PEP 440 version, but '0.1.BUILD-ID' is not
Failed to build carrier
ERROR: Could not build wheels for carrier, which is required to install pyproject.toml-based projects
make: *** [Makefile:7: install] Error 1

As you can see in the pipeline file the PDAL binaries are installed via apt-get. Any suggestion is appreciated to resolve this both locally in a venv environment and for deployment.

[hobu]

Could not build wheels for carrier,

The error is about carrier, not PDAL. Is it a complaint about how PDAL does its metadata? If so, our conda packages would not be building. I don't doubt there's something going on, but there's not enough here for me to go on. If you can provide a Dockerfile that demonstrates the issue cleanly, I would have something to start from.

[shayannikoohemat]

@hobu sorry for the late reply. The build error about the carrier package is because python-pdal can't be build. Using conda, I don't have a problem it works but I want to make it work in a virtual environment. To reproduce the error in Ubuntu20.04 and Ubuntu22.04: installing pdal sudo apt-get install pdal

create a new venv environment with: python3.9: /usr/bin/python3.9 -m venv pdal-test39

then activate the environment and run: pip3 install --upgrade pip pip install pdal --verbose

Since you asked for a clean Docker, here you are:

FROM ubuntu:20.04

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update && apt-get upgrade -y && \
    apt-get install -y \
    apt-utils \
    python3.9 \
    python3-pip \
    python3-venv \
    python3-virtualenv \
    make \
    cmake \
    pdal

WORKDIR /opt

RUN python3 -m venv venv-pypdal

Build the docker file: docker build -t pypdal:1 .

Run the docker file: docker run -it pypdal:1

Check pdal installation: pdal --version

--------------------------------------------------------------------------------
pdal 2.0.1 (git-version: Release)
--------------------------------------------------------------------------------

Activate the VENV: source venv-pypdal/bin/activate

Install python-pdal: pip3 install --upgrade pip && pip install pdal --verbose

After running above command in the docker, I get the following error: (I copied the last part)

...
LEPATH=/tmp/pip-build-env-3mddi3jd/overlay/lib/python3.8/site-packages/ninja/data/bin/ninja -DCMAKE_BUILD_TYPE:STRING=Release
    Source directory:
      /tmp/pip-install-qy_l9zmy/pdal_ea52fa9ffafc4789ba802137e8f0fd83
    Working directory:
      /tmp/pip-install-qy_l9zmy/pdal_ea52fa9ffafc4789ba802137e8f0fd83/_skbuild/linux-x86_64-3.8/cmake-build
  Please see CMake's output for more information.
  error: subprocess-exited-with-error

  × Building wheel for pdal (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /opt/venv-pypdal/bin/python3 /opt/venv-pypdal/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmp7qjgkvok
  cwd: /tmp/pip-install-qy_l9zmy/pdal_ea52fa9ffafc4789ba802137e8f0fd83
  Building wheel for pdal (pyproject.toml) ... error
  ERROR: Failed building wheel for pdal
Failed to build pdal
ERROR: Could not build wheels for pdal, which is required to install pyproject.toml-based projects

[hobu]

pdal 2.0.1 (git-version: Release)

The previous logs did not have this information. This version is going to be way too old. I should think you'll need to be at least PDAL 2.2 for the new PDAL Python bindings to work with it.

PDAL has been removed from Debian/Ubuntu for 22.04 going forward. Conda, Docker, a different distribution, or build-you-own will be needed.

[shayannikoohemat]

Thanks for the above. Can I install pdal 2.2 or a higher version for Debian with apt-get on Ubuntu 20.04? (for packaging my python code I don't use Docker or Conda)

[hobu]

Can I install pdal 2.2 or a higher version for Debian with apt-get on Ubuntu 20.04?

Not from common package repositories. As I mentioned, PDAL was removed from Debian as a package. If Debian PDAL packages are a hard requirement for you, please step forward as a Debian maintainer.