pgpointcloud / pointcloud

A PostgreSQL extension for storing point cloud (LIDAR) data.
https://pgpointcloud.github.io/pointcloud/
Other
388 stars 107 forks source link

PDAL writers.pgpointcloud - duplicate dimensions on direct .dll call #281

Closed losbaltica closed 2 years ago

losbaltica commented 3 years ago

It is the continuation of issue originally raised against CAPI. They suggest contacting directly to PDAL team as it might be .dll bug. I am currently writing a small C# program to upload las to Postgres database using PDAL writers.pgpointcloud. My JSON pipeline:

{
  "pipeline": [
    {
      "type": "readers.las",
      "filename": "sample.laz",
      "extra_dims": "HeightAboveGround=float32"
    },
    {
      "type": "filters.range",
      "limits": "Classification[5:5], Classification[10:10], Classification[14:14], Classification[17:17]"
    },
    {
      "type": "filters.chipper",
      "capacity": 600
    },
    {
      "type": "writers.pgpointcloud",
      "connection": "host='localhost' dbname='test' user='postgres' password='123456' port='5432' ",
      "schema": "test",
      "table": "sample",
      "compression": "dimensional",
      "output_dims": "X, Y, Z, Classification, GpsTime, HeightAboveGround"
    }
  ]
} 

When executing Pipeline through EXE I am getting a different number of dimensions to compare to executing pipeline from CAPI (direct .dll call). image

Other pipelines which I was testing was working fine. Is just something with Postgres import. Not sure if the issue is with CAPI or maybe is deeper in PDAL.dll's. However, when I run pdal.exe from the same folder with pdal.dlls, with the same pipeline, dimensions seem to not be duplicated. The other interesting behaviour I noticed is that when I query for a single dimension in PG, I am getting a single correct result. A number of points are correct too. Maybe there is some metadata underneath which gets corrupted when calling pipeline directly from CAPI or rather dll.

SELECT pc_get(PC_Explode(pa), 'HeightAboveGround') pt
FROM test.sample LIMIT 1;

image

When I query pointcloud_formats XML schema I can see the dims duplications. At what stage XML is being generated?

SELECT * FROM pointcloud_formats;
<?xml version="1.0" encoding="UTF-8"?>
<pc:PointCloudSchema xmlns:pc="http://pointcloud.org/schemas/PC/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
 <pc:dimension>
  <pc:position>1</pc:position>
  <pc:size>1</pc:size>
  <pc:description>ASPRS classification.  0 for no classification.  See LAS specification for details.</pc:description>
  <pc:name>Classification</pc:name>
  <pc:interpretation>uint8_t</pc:interpretation>
  <pc:active>true</pc:active>
 </pc:dimension>
 <pc:dimension>
  <pc:position>2</pc:position>
  <pc:size>1</pc:size>
  <pc:description>ASPRS classification.  0 for no classification.  See LAS specification for details.</pc:description>
  <pc:name>Classification</pc:name>
  <pc:interpretation>uint8_t</pc:interpretation>
  <pc:active>true</pc:active>
 </pc:dimension>
 <pc:dimension>
  <pc:position>3</pc:position>
  <pc:size>8</pc:size>
  <pc:description>GPS time that the point was acquired</pc:description>
  <pc:name>GpsTime</pc:name>
  <pc:interpretation>double</pc:interpretation>
  <pc:active>true</pc:active>
 </pc:dimension>
 <pc:dimension>
  <pc:position>4</pc:position>
  <pc:size>8</pc:size>
  <pc:description>GPS time that the point was acquired</pc:description>
  <pc:name>GpsTime</pc:name>
  <pc:interpretation>double</pc:interpretation>
  <pc:active>true</pc:active>
 </pc:dimension>
 <pc:dimension>
  <pc:position>5</pc:position>
  <pc:size>4</pc:size>
  <pc:description>Height Above Ground</pc:description>
  <pc:name>HeightAboveGround</pc:name>
  <pc:interpretation>float</pc:interpretation>
  <pc:active>true</pc:active>
 </pc:dimension>
 <pc:dimension>
  <pc:position>6</pc:position>
  <pc:size>4</pc:size>
  <pc:description>Height Above Ground</pc:description>
  <pc:name>HeightAboveGround</pc:name>
  <pc:interpretation>float</pc:interpretation>
  <pc:active>true</pc:active>
 </pc:dimension>
 <pc:dimension>
  <pc:position>7</pc:position>
  <pc:size>8</pc:size>
  <pc:description>X coordinate</pc:description>
  <pc:name>X</pc:name>
  <pc:interpretation>double</pc:interpretation>
  <pc:active>true</pc:active>
 </pc:dimension>
 <pc:dimension>
  <pc:position>8</pc:position>
  <pc:size>8</pc:size>
  <pc:description>X coordinate</pc:description>
  <pc:name>X</pc:name>
  <pc:interpretation>double</pc:interpretation>
  <pc:active>true</pc:active>
 </pc:dimension>
 <pc:dimension>
  <pc:position>9</pc:position>
  <pc:size>8</pc:size>
  <pc:description>Y coordinate</pc:description>
  <pc:name>Y</pc:name>
  <pc:interpretation>double</pc:interpretation>
  <pc:active>true</pc:active>
 </pc:dimension>
 <pc:dimension>
  <pc:position>10</pc:position>
  <pc:size>8</pc:size>
  <pc:description>Y coordinate</pc:description>
  <pc:name>Y</pc:name>
  <pc:interpretation>double</pc:interpretation>
  <pc:active>true</pc:active>
 </pc:dimension>
 <pc:dimension>
  <pc:position>11</pc:position>
  <pc:size>8</pc:size>
  <pc:description>Z coordinate</pc:description>
  <pc:name>Z</pc:name>
  <pc:interpretation>double</pc:interpretation>
  <pc:active>true</pc:active>
 </pc:dimension>
 <pc:dimension>
  <pc:position>12</pc:position>
  <pc:size>8</pc:size>
  <pc:description>Z coordinate</pc:description>
  <pc:name>Z</pc:name>
  <pc:interpretation>double</pc:interpretation>
  <pc:active>true</pc:active>
 </pc:dimension>
 <pc:metadata>
  <Metadata name="compression" type="string">dimensional</Metadata>
 </pc:metadata>
 <pc:orientation>point</pc:orientation>
 <pc:version>1.3</pc:version>
</pc:PointCloudSchema>

Does anyone know the reason why the dimensions get duplicated or observe similar behave?

pblottiere commented 2 years ago

The underlying issue seems to be in PDAL.