Visualization and segmentation code is not working

[niranjanreddy891]

I am using win-7 64 bit with python-3.6 installed. When I am running the code, I am not getting any error but the problem is visualization window is not appearing. Can anyone help me in this issue? :)

[davidcaron]

Can I see the code you ran? It's likely a VTK issue but I'm surprised you're not getting an error message.

[niranjanreddy891]

Thanks for the reply @davidcaron This is the code for visualization

import os
import numpy as np
import pytest

import pclpy
from pclpy import pcl
import pclpy.view.vtk

def test_data(*args):
    return os.path.join("test_data", *args)

def test_cloud_viewer():
    pc = pclpy.io.read(test_data("street.las"), "PointXYZRGBA")
    viewer = pcl.visualization.CloudViewer("viewer")
    # viewer.showCloud(pc, "hi")

def test_pcl_visualizer_simple():
    pc = pclpy.io.read(test_data("street.las"), "PointXYZRGBA")
    viewer = pcl.visualization.PCLVisualizer("viewer")
    viewer.setBackgroundColor(0, 0, 0)
    rgb = pcl.visualization.PointCloudColorHandlerRGBAField.PointXYZRGBA(pc)
    viewer.addPointCloud(pc, rgb, "sample cloud")
    viewer.setPointCloudRenderingProperties(0, 3, "sample cloud")
    viewer.addCoordinateSystem(1.0)
    viewer.initCameraParameters()
    # while not viewer.wasStopped():
    #     viewer.spinOnce(10, False)

def test_viewer_intensity():
    pc = pclpy.io.read(test_data("street.las"), "PointXYZI")
    viewer = pclpy.view.vtk.Viewer(pc)
    # viewer.show()

[davidcaron]

This is the code for the automated tests, did you try uncommenting the lines where the cloud is actually displayed? I skip these lines in automated tests because the opened window would block the other tests.

I added some notes to the source:

https://github.com/davidcaron/pclpy/blob/master/pclpy/tests/test_visualization.py

What issue are you getting with the region growing segmentation?

[davidcaron]

So you try to execute this code as is? The functions aren't called, this is meant to be executed by the pytest test suite.

Try adding test_pcl_visualizer_simple() at the bottom to actually call the test

[niranjanreddy891]

I added test_pcl_visualizer_simple() at the end, I am getting a visualization window without any output. It shows just a simple xyz axis.

[davidcaron]

This is actually the default behavior of the PCL visualiser. Refer to the PCL documentation.

Use the R key to see the whole cloud, the F key to set rotation point, the H key to print commands, etc.

[niranjanreddy891]

Ya now it works. Is it possible to select ROI in the output window? I am very curious to know

[davidcaron]

You mean select points? You'd have to refer to the PCL documentation for that. The API in pclpy should be very similar.

I'll explore these viewer customizations myself when I get the chance, but for now I mostly focused on the binding itself.

[niranjanreddy891]

Thanks for the response. Hope this library gets more features in future. :+1:

[davidcaron]

which of the 2 test functions are you calling?

[niranjanreddy891]

I am calling the second one def test_region_growing():

[niranjanreddy891]

This is the code

import math
import os
import numpy as np
import pytest

import pclpy
from pclpy import pcl

def test_data(*args):
    return os.path.join("test_data", *args)

def make_pt(x, y, z):
    pt = pcl.point_types.PointXYZRGBA()
    pt.x = x
    pt.y = y
    pt.z = z
    return pt

def test_region_growing():
    pc = pclpy.io.read(test_data("E:/Lidar-practice/new-files/las14_type7_ALS_RIEGL_Q680i.las"), "PointXYZRGBA")
    rg = pcl.segmentation.RegionGrowing.PointXYZRGBA_Normal()
    rg.setInputCloud(pc)
    normals_estimation = pcl.features.NormalEstimationOMP.PointXYZRGBA_Normal()
    normals_estimation.setInputCloud(pc)
    normals = pcl.PointCloud.Normal()
    normals_estimation.setRadiusSearch(0.1)
    normals_estimation.compute(normals)
    rg.setInputNormals(normals)

    clouds = []
    for n, th in enumerate([0.01, 10, 100, 1000]):
        rg.setMaxClusterSize(10000000)
        rg.setMinClusterSize(100)
        rg.setNumberOfNeighbours(15)
        rg.setSmoothnessThreshold(5 / 180 * math.pi)
        rg.setCurvatureThreshold(20)
        rg.setResidualThreshold(th)
        clusters = pcl.vectors.PointIndices()
        rg.extract(clusters)
        cloud = rg.getColoredCloud()
        clouds.append(cloud)
        pclpy.io.write(cloud, test_data("E:/Lidar-practice/new-files/oo.las" % n))

[davidcaron]

Are you familiar with the region growing segmentation? You will have to set the parameters according to your input cloud.

Try this code, but make sure to use parameters according to your point cloud. Refer to the pcl documentation: http://pointclouds.org/documentation/tutorials/region_growing_segmentation.php

import math
import pclpy
from pclpy import pcl

pc = pclpy.io.las.read("E:/Lidar-practice/new-files/las14_type7_ALS_RIEGL_Q680i.las", "PointXYZRGBA")
rg = pcl.segmentation.RegionGrowing.PointXYZRGBA_Normal()
rg.setInputCloud(pc)
normals_estimation = pcl.features.NormalEstimationOMP.PointXYZRGBA_Normal()
normals_estimation.setInputCloud(pc)
normals = pcl.PointCloud.Normal()
normals_estimation.setRadiusSearch(0.1)
normals_estimation.compute(normals)
rg.setInputNormals(normals)

rg.setMaxClusterSize(10000000)
rg.setMinClusterSize(100)
rg.setNumberOfNeighbours(15)
rg.setSmoothnessThreshold(5 / 180 * math.pi)
rg.setCurvatureThreshold(20)
rg.setResidualThreshold(10)
clusters = pcl.vectors.PointIndices()
rg.extract(clusters)
cloud = rg.getColoredCloud()
pclpy.io.las.write(cloud, "E:/Lidar-practice/new-files/oo.las")

[davidcaron]

At which line are you getting this error?

If you get this error when reading the file, try using:

pc = pclpy.io.las.read("your_las.las", "PointXYZ")

Instead of:

pc = pclpy.io.las.read("your_las.las", "PointXYZRGBA")

Because your las file probably doesn't contain color.

[davidcaron]

Region growing is usually not very good at segmenting the ground, depending on what the ground looks like. Try the ProgressiveMorphologicalFilter.

Also, the CSF plugin in CloudCompare can get good results.

[niranjanreddy891]

Can you suggest me any other instead of region growing? :)

[niranjanreddy891]

Regarding visualization, as you told before to use this command pc = pclpy.io.las.read("your_las.las", "PointXYZ") , It throws this error

rgb = pcl.visualization.PointCloudColorHandlerRGBAField.PointXYZRGBA(pc)
TypeError: __init__(): incompatible constructor arguments. The following argument types are supported:
    1. pclpy.pcl.visualization.PointCloudColorHandlerRGBAField.PointXYZRGBA()
    2. pclpy.pcl.visualization.PointCloudColorHandlerRGBAField.PointXYZRGBA(cloud: pclpy.pcl.PointCloud.PointXYZRGBA)

Invoked with: <pclpy.pcl.PointCloud.PointXYZ object at 0x00000000126DF618>

[davidcaron]

Creating a RGBA color handler for a point cloud without color will obviously not work. Again, refer to the PCL documentation.

Try the simple CloudViewer.

pc = pclpy.io.las.read("your_cloud.las", "PointXYZ")
viewer = pcl.visualization.CloudViewer("viewer")
viewer.showCloud(pc, "cloud")

while not viewer.wasStopped(10):
    pass

[niranjanreddy891]

When I tried the sample code which you mentioned, the point cloud looks just like a small dot. i used F to zoom in, but it is still showing a small dot.

[davidcaron]

F itself won't zoom, try zooming with the mouse wheel. Can you send me your cloud?

[davidcaron]

Hard to tell without having the file...

[niranjanreddy891]

This is the file

new-las-3.zip

[davidcaron]

You have large coordinates and PCL uses float32 only, not float64.

You can use the pclpy.io.las.get_offset function to read the offset from the las file and pass this value to the las reader or the las writer like this:

import pclpy
from pclpy import pcl

p = r"C:\Users\David Caron\Downloads\new-las-3.las"

offset = pclpy.io.las.get_offset(p)
pc = pclpy.io.las.read(p, "PointXYZRGBA", xyz_offset=offset)
viewer = pcl.visualization.CloudViewer("viewer")
viewer.showCloud(pc, "cloud")

while not viewer.wasStopped(10):
    pass

This will temporarily shift your point cloud coordinates.

[niranjanreddy891]

How to fit the output in the visualization window when we load the file? Without zooming, I should fit the see the output. How can we do it?

[davidcaron]

resetCamera() should do the trick

[niranjanreddy891]

Adding the resetCamera function, didn't worked.

import pclpy
from pclpy import pcl

p = r"C:/Users/Niranjan/Downloads/tutorial1/stadium.las"

offset = pclpy.io.las.get_offset(p)
pc = pclpy.io.las.read(p, "PointXYZRGBA", xyz_offset=offset)
viewer = pcl.visualization.CloudViewer("viewer")
viewer.showCloud(pc, "cloud")
viewer.resetCamera()

while not viewer.wasStopped(10):
    pass