Python driver not found after installing deps for a Dockerfile

[Nogbit]

Im trying to create a Docker image that has Python + pdal itself and pdal python package.

So I exec in

docker run -it  pdal/pdal  /bin/bash

Then install the deps

apt-get update && \
apt-get install python3 python3-pip -y && \
python3 -m pip install cython numpy packaging scikit-build

And get the following error (this is after running the pip install above the second time, hence the "already installed" messages.)

root@9e12f616ca0a:/# python3 -m pip install cython numpy packaging scikit-build pdal
Requirement already satisfied: cython in /usr/local/lib/python3.6/dist-packages
Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages
Requirement already satisfied: packaging in /usr/local/lib/python3.6/dist-packages
Requirement already satisfied: scikit-build in /usr/local/lib/python3.6/dist-packages
Collecting pdal
  Using cached https://files.pythonhosted.org/packages/3e/c3/2f5ec61e5c38c2552111869240ae54872507a806793d0fa9289344be2c72/PDAL-2.3.7.tar.gz
Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.6/dist-packages (from packaging)
Requirement already satisfied: wheel>=0.29.0 in /usr/lib/python3/dist-packages (from scikit-build)
Requirement already satisfied: distro in /usr/local/lib/python3.6/dist-packages (from scikit-build)
Requirement already satisfied: setuptools>=28.0.0; python_version >= "3" in /usr/lib/python3/dist-packages (from scikit-build)
Building wheels for collected packages: pdal
  Running setup.py bdist_wheel for pdal ... error
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-mebvq7s_/pdal/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpnnh5hmd3pip-wheel- --python-tag cp36:
  /usr/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'long_description_content_type'
    warnings.warn(msg)
  Traceback (most recent call last):
    File "/usr/local/lib/python3.6/dist-packages/skbuild/setuptools_wrap.py", line 560, in setup
      cmkr = cmaker.CMaker(cmake_executable)
    File "/usr/local/lib/python3.6/dist-packages/skbuild/cmaker.py", line 95, in __init__
      self.cmake_version = get_cmake_version(self.cmake_executable)
    File "/usr/local/lib/python3.6/dist-packages/skbuild/cmaker.py", line 82, in get_cmake_version
      "Problem with the CMake installation, aborting build. CMake executable is %s" % cmake_executable)

  Problem with the CMake installation, aborting build. CMake executable is cmake

  ----------------------------------------
  Failed building wheel for pdal
  Running setup.py clean for pdal
Failed to build pdal
Installing collected packages: pdal
  Running setup.py install for pdal ... error
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-mebvq7s_/pdal/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-h31pzegv-record/install-record.txt --single-version-externally-managed --compile:
    /usr/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'long_description_content_type'
      warnings.warn(msg)
    Traceback (most recent call last):
      File "/usr/local/lib/python3.6/dist-packages/skbuild/setuptools_wrap.py", line 560, in setup
        cmkr = cmaker.CMaker(cmake_executable)
      File "/usr/local/lib/python3.6/dist-packages/skbuild/cmaker.py", line 95, in __init__
        self.cmake_version = get_cmake_version(self.cmake_executable)
      File "/usr/local/lib/python3.6/dist-packages/skbuild/cmaker.py", line 82, in get_cmake_version
        "Problem with the CMake installation, aborting build. CMake executable is %s" % cmake_executable)

    Problem with the CMake installation, aborting build. CMake executable is cmake

    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-mebvq7s_/pdal/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-h31pzegv-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-mebvq7s_/pdal/

[Nogbit]

I think its because I wrongly assumed cmake was already installed, will do that...need a lunch break first ;-) Most likely need gcc-multilib.

[Nogbit]

Using @hobu Dockerfile from this post https://github.com/PDAL/python/issues/56

However, the --drivers is not listing Python. I'm able to do an import pdal without error on a python script.

FROM pdal/pdal:latest
RUN apt-get update -y && apt-get install python3-pip -y
RUN sed -i 's+opt/conda/envs/pdal+usr+g' /usr/lib/cmake/PDAL/PDALConfig.cmake
RUN sed -i 's+opt/conda/envs/pdal+usr+g' /usr/lib/cmake/PDAL/PDALTargets.cmake
RUN pip3 install pip --upgrade
RUN pip3 install numpy packaging cython scikit-build
RUN pip3 install  PDAL
ENV PDAL_DRIVER_PATH=/usr/local/lib
RUN pdal --drivers

Error when running a pipeline with a python filter

PDAL: Couldn't create filter stage of type 'filters.python'.
You probably have a version of PDAL that didn't come with a plugin
you're trying to load.  Please see the FAQ at https://pdal.io/faq

Drivers

============================ ===================================================
Name                         Description
============================ ===================================================
filters.approximatecoplanar  Estimates the planarity of a neighborhood of
                             points using eigenvalues.
filters.assign               Assign values for a dimension range to a specified
                             value.
filters.chipper              Organize points into spatially contiguous,
                             squarish, and non-overlapping chips.
filters.cluster              Extract and label clusters using Euclidean
                             distance.
filters.colorinterp          Assigns RGB colors based on a dimension and a ramp
filters.colorization         Fetch and assign RGB color information from a
                             GDAL-readable datasource.
filters.covariancefeatures   Filter that calculates local features based on the
                             covariance matrix of a point's neighborhood.
filters.crop                 Filter points inside or outside a bounding box or
                             a polygon
filters.csf                  Cloth Simulation Filter (Zhang et al., 2016)
filters.dbscan               DBSCAN Clustering.
filters.decimation           Rank decimation filter. Keep every Nth point
filters.delaunay             Perform Delaunay triangulation of a pointcloud
filters.dem                  Filter points about an elevation surface
filters.divider              Divide points into approximately equal sized
                             groups based on a simple scheme
filters.eigenvalues          Returns the eigenvalues for a given point, based
                             on its k-nearest neighbors.
filters.elm                  Marks low points as noise.
filters.estimaterank         Computes the rank of a neighborhood of points.
filters.faceraster           Face Raster Filter
filters.ferry                Copy data from one dimension to another.
filters.fps                  Farthest point sampling filter
filters.greedyprojection     Greedy Triangulation filter
filters.groupby              Split data categorically by dimension.
filters.hag_delaunay         Computes height above ground using delaunay
                             interpolation of ground returns.
filters.hag_dem              Computes height above ground using a DEM raster.
filters.hag_nn               Computes height above ground using
                             nearest-neighbor ground-classified returns.
filters.head                 Return N points from beginning of the point cloud.
filters.hexbin               Tessellate the point's X/Y domain and determine
                             point density and/or point boundary.
filters.icp                  Iterative Closest Point (ICP) registration.
filters.info                 Gather basic info about points.
filters.iqr                  Interquartile Range Filter
filters.litree               Li Tree Filter
filters.lloydkmeans          Extract and label clusters using K-means (Lloyd's
                             algorithm).
filters.locate               Return a single point with min/max value in the
                             named dimension.
filters.lof                  LOF Filter
filters.mad                  Median Absolute Deviation Filter
filters.merge                Merge data from two different readers into a
                             single stream.
filters.miniball             Miniball (Kutz et al., 2003)
filters.mongo                Pass only points that pass a logic filter.
filters.mortonorder          Morton or z-order sorting of points. See
                             http://en.wikipedia.org/wiki/Z-order_curve for
                             more detail.
filters.neighborclassifier   Re-assign some point attributes based KNN voting
filters.nndistance           NN-Distance Filter
filters.normal               Normal Filter
filters.optimalneighborhood  OptimalNeighborhood Filter
filters.outlier              Outlier removal
filters.overlay              Assign values to a dimension based on the extent
                             of an OGR-readable data source or an OGR SQL
                             query.
filters.planefit             Plane Fit (Kutz et al., 2003)
filters.pmf                  Progressive morphological filter
filters.poisson              Poisson Surface Reconstruction Filter
filters.projpipeline         Transform coordinates using Proj pipeline string,
                             WKT2 coordinate operations or URN definition
filters.radialdensity        RadialDensity Filter
filters.randomize            Randomize points in a view.
filters.range                Pass only points given a dimension/range.
filters.reciprocity          Returns the percentage of neighbors that do NOT
                             have the query point as a neighbor
filters.relaxationdartthrowing Subsampling filter
filters.reprojection         Reproject data using GDAL from one coordinate
                             system to another.
filters.returns              Split data by return order
filters.sample               Subsampling filter
filters.separatescanline     Split data by scan line.
filters.shell                Execute a shell operation inline with PDAL
                             pipeline steps
filters.skewnessbalancing    Bartels & Wei Skewness Balancing
filters.smrf                 Simple Morphological Filter (Pingel et al., 2013)
filters.sort                 Sort data based on a given dimension.
filters.splitter             Split data based on a X/Y box length.
filters.stats                Compute statistics about each dimension (mean,
                             min, max, etc.)
filters.streamcallback       Provide a hook for a simple point-by-point
                             callback.
filters.tail                 Return N points from end of the point cloud.
filters.transformation       Transform each point using a 4x4 transformation
                             matrix
filters.voxelcenternearestneighbor Voxel Center Nearest Neighbor Filter
filters.voxelcentroidnearestneighbor Voxel Centroid Nearest Neighbor Filter
filters.voxeldownsize        First Entry Voxel Filter
readers.bpf                  "Binary Point Format" (BPF) reader support. BPF is
                             a simple DoD and research format that is used by
                             some sensor and processing chains.
readers.ept                  EPT Reader
readers.faux                 Faux Reader
readers.gdal                 Read GDAL rasters as point clouds.
readers.ilvis2               ILVIS2 Reader
readers.las                  ASPRS LAS 1.0 - 1.4 read support. LASzip support
                             is also enabled through this driver if LASzip was
                             found during compilation.
readers.memoryview           Memory View Reader
readers.obj                  Obj Reader
readers.optech               Optech reader support.
readers.pcd                  Read data in the Point Cloud Library (PCL) format.
readers.ply                  Read ply files.
readers.pts                  Pts Reader
readers.qfit                 QFIT Reader
readers.sbet                 SBET Reader
readers.terrasolid           TerraSolid Reader
readers.text                 Text Reader
readers.tindex               TileIndex Reader
writers.bpf                  "Binary Point Format" (BPF) writer support. BPF is
                             a simple DoD and research format that is used by
                             some sensor and processing chains.
writers.ept_addon            EPT Writer
writers.gdal                 Write a point cloud as a GDAL raster.
writers.gltf                 Gltf Writer
writers.las                  ASPRS LAS 1.0 - 1.4 writer. LASzip support is also
                             available if enabled at compile-time. Note that
                             LAZ does not provide LAS 1.4 support at this time.
writers.null                 Null writer. Provides a sink for points in a
                             pipeline. It's the same as sending pipeline output
                             to /dev/null.
writers.ogr                  Write a point cloud as a set of OGR
                             points/multipoints
writers.pcd                  Write data in the Point Cloud Library (PCL)
                             format.
writers.ply                  ply writer
writers.raster               Write a raster.
writers.sbet                 SBET Writer
writers.text                 Text Writer
============================ ===================================================

[Nogbit]

This works for now

FROM pdal/pdal

RUN apt-get update -y && apt-get install python3-pip -y

RUN pip3 install pip --upgrade && \
    pip3 install numpy packaging cython scikit-build PDAL

# ideally, have this be the output of "find /usr/local -name pdal"
ENV PDAL_DRIVER_PATH=/usr/local/lib/python3.6/dist-packages/pdal

And to build/run

docker build . -t pdal-local
docker run pdal-local pdal --drivers |grep python

filters.python Manipulate data using inline Python