Add AABB and better documentation to `pcl::visualizer`

[yiakwy]

Background

I am trying to implement a segmentation algorithm with the aid of PCL and to visualize them locally in my c++ projects. I have already implemented a cross-platform WebGL based "game engine" to visualize pcd data layer but it will save my time if I can have rich scene management tool locally in my c++ projects.

I have scanned through sources code of "PCL/visualization/pcl_visualizer.h" and found that it brings much more functionalities than I expected:

1. add a camera
2. multi viewports
3. draw primitive geometries
4. event handling

But I cannot find details on how to use them besides the scope of source codes. I am familiar with openGL3.0 but not that good at Qt or VTK UI for desktop projects (I am good at HTML5 based solutions for the cross-platform product). Hence I would like to reuse any codes written by PCL.

It is not easy. For example, if I want to add a grid, do I need to use addLine to draw them from scratch?

Or another example, If I want add an AABB, do need to add routines like using OpenGL? Or, even better, you cool guys have done something for me already!

I have little idea on how to use or extend something like PCL::visualization::PCLVisualizerInteractorStyle which might be important to visualizer.

Look at some toolkits installation file:

# pcl_outofcore_process
PCL_ADD_EXECUTABLE(pcl_outofcore_process "${SUBSYS_NAME}" outofcore_process.cpp)
target_link_libraries(pcl_outofcore_process pcl_common pcl_filters pcl_io pcl_octree pcl_outofcore)

PCL_ADD_EXECUTABLE(pcl_outofcore_print "${SUBSYS_NAME}" outofcore_print.cpp)
target_link_libraries(pcl_outofcore_print pcl_common pcl_filters pcl_io pcl_octree pcl_outofcore)

if(NOT VTK_FOUND)
  set(DEFAULT FALSE)
  set(REASON "VTK was not found.")
else(NOT VTK_FOUND)
  set(DEFAULT TRUE)
  set(REASON)
  set(VTK_USE_FILE "${VTK_USE_FILE}" CACHE INTERNAL "VTK_USE_FILE")
  include("${VTK_USE_FILE}")
  include_directories("${CMAKE_CURRENT_SOURCE_DIR}/include")

  if("${VTK_MAJOR_VERSION}.${VTK_MINOR_VERSION}" VERSION_GREATER "5.8")
    # pcl_outofcore_viewer uses some functions not present in vtk 5.8

    set(srcs outofcore_viewer.cpp
           ../src/visualization/camera.cpp
           ../src/visualization/common.cpp
           ../src/visualization/object.cpp
           ../src/visualization/grid.cpp
           ../src/visualization/outofcore_cloud.cpp
           ../src/visualization/scene.cpp
           ../src/visualization/viewport.cpp)

    # pcl_outofcore_viewer
    PCL_ADD_EXECUTABLE_OPT_BUNDLE(pcl_outofcore_viewer "${SUBSYS_NAME}" ${srcs})
    target_link_libraries(pcl_outofcore_viewer pcl_common pcl_io pcl_outofcore pcl_visualization pcl_octree pcl_filters)

  endif("${VTK_MAJOR_VERSION}.${VTK_MINOR_VERSION}" VERSION_GREATER "5.8")
endif(NOT VTK_FOUND)

Where I can find the usage of them?

Subject

Detailed information on existing functionalities to enrich the visualizer like:

1. adding AABB
2. adding grid
3. using scene graph inside the PCL visualizer
4. provided that scene graph available, event handler enhancement for each object added into scene graph in PCL visualizer.
5. I do really need some examples, detailed documentation and starter instead of reading source codes because adding functionalities into PCL visualizer, exceeds beyond my duties or performance in this project.

Guide to answer the question

If we don't have such utilities do those things, I will consider contributing the relevant codes into the PCL code base even though It has little influence upon the main line of the project.

Current behavior

I cannot find official documentation and starter projects to verify whether the functionalities available.

Expected behavior

Visualizer seems to be implemented with great efforts from the PCL developers, we need finer details to utilize the codes they wrote.

@clockwise9 @rbrusu @stevefox @elfring

[taketwo]

There are several tutorials covering the visualization module: http://pointclouds.org/documentation/tutorials/#visualization-tutorial

[yiakwy]

Hi @taketwo , I do have read the tutorial and part of source codes of visualizer carefully. That's why I found that existing tutorial is not well-suited to answer my questions in Subject.

The tutorial is very basic and hasn't exposed the design philosophy, framework structure and part of functions distributed in the visualizer.

I don't want to waste your time if I didn't have a research on it. ^ ^

[SergioRAgostinho]

We're mainly a point cloud processing library. Our visualization module is there to assist visualizing the results of data we process. VTK does all the heavy lifting for us. We wrap things in a way which makes it easier to our user to visualize PCL's native types without having to worry about VTK.

Detailed information on existing functionalities to enrich the visualizer like:

1. adding AABB

This one can be useful.

2. adding grid

Too specific of your use case.

3. using scene graph inside the PCL visualizer

I don't want to expose that. You can derive a class from the visualizer and access the underlying VTK objects if you need access to the scene graph.

4. provided that scene graph available, event handler enhancement for each object added into scene graph in PCL visualizer.

Same answer as in 3.

5. I do really need some examples, detailed documentation and starter instead of reading source codes because adding functionalities into PCL visualizer, exceeds beyond my duties or performance in this project.

We try to cover the most common use cases. Yours is just too specific. With your current needs you should not rely on our visualization module, you really should use VTK directly.

[yiakwy]

Hi, @SergioRAgostinh thanks for your comment. I thought twice about them.

I don't want to expose that. You can derive a class from the visualizer and access the underlying VTK objects if you need access to the scene graph.

Could you show me an example to guide me along the way? I believe that this enables me to do much more interesting things.

Too specific for your answer.

I draw it for lidar distance measurement visualization and objects segmentation. Here is the algorithm:

Split the ground into grids. We perform objects detection upon each block. Finally, I use compressed union find algorithms to aggregate blocks into groups.

I understand that PCL has already encapsulated some operations upon VTK actor and viewer. I am new to VTK and I would like to reuse PCL codes as much as possible. For example, I have scanned through codes in

https://github.com/PointCloudLibrary/pcl/blob/master/outofcore/include/pcl/outofcore/visualization

Wish that it helps.

@taketwo @SergioRAgostinho , I also noticed that "visualization" is implemented in a way very different from “outforce”. And the later one conceptually shares more graphics characteristics and easy to be extended.

[yiakwy]

For example, we have a very large file to hold PCLVisualizer. Here is my proposal:

1. using public interfaces to pass vtkRenderer object (pcl v 1.9)

PCLVisualizer (vtkSmartPointer ren, vtkSmartPointer wind, const std::string &name = "", const bool create_interactor = true);

https://github.com/PointCloudLibrary/pcl/blob/master/visualization/include/pcl/visualization/pcl_visualizer.h, starting from Line 122.

The codes also indicate that we have two additional controls vtkRenderWindow and vtkRenderCollection (if multiple viewports are available), Line 1891 ~ 1896.

2. Other relevant methods like setupRenderer, setupRenderer and members are private and we cannot access it without changing the source code.

3. VTK also provides additional concepts which have been used by the PCLVisualizer implementation. They compromise of vtkRenderWindowInteractor, vtkRenderWindow and so on. They make the job complicated.

Instead of using

pcl::visualization::PCLVisualizer::PCLVisualizer (int &argc, char **argv, const std::string &name, PCLVisualizerInteractorStyle* style, const bool create_interactor)
  : interactor_ ()
  , update_fps_ (vtkSmartPointer<FPSCallback>::New ())
  , stopped_ ()
  , timer_id_ ()
  , exit_main_loop_timer_callback_ ()
  , exit_callback_ ()
  , rens_ (vtkSmartPointer<vtkRendererCollection>::New ())
  , win_ (vtkSmartPointer<vtkRenderWindow>::New ())
  , style_ (style)
  , cloud_actor_map_ (new CloudActorMap)
  , shape_actor_map_ (new ShapeActorMap)
  , coordinate_actor_map_ (new CoordinateActorMap)
  , camera_set_ ()
  , camera_file_loaded_ (false)
{
  vtkSmartPointer<vtkRenderer> ren = vtkSmartPointer<vtkRenderer>::New ();
  setupRenderer (ren);
  setupFPSCallback (ren);
  setupRenderWindow (name);
  setupStyle ();
  setupCamera (argc, argv);

  if(!camera_set_ && !camera_file_loaded_)
    setDefaultWindowSizeAndPos ();

  if (create_interactor)
    createInteractor ();

  //window name should be reset due to its reset somewhere in camera initialization
  win_->SetWindowName (name.c_str ());
}

Is there any examples for me to use the internal ren to modify the underlying renderer in pcl v1.8?

UPDATE ON 2018/12/12

My current solution for PCL v1.8 is to use vtkRendererWindow and carefully operate on rendererCollection.

It seems that from PCL V1.9, the developers try to expose the components they used. I also figured out some problems in the implementation of outforces. The right way to implement a scene graph is to write a geometry (Geometry) which delegates all the operations to exchange data with GPU and inherits all the tree methods so that we can compute global and local transformations.

[yiakwy]

I also noticed that there is a huge gap between PCL v 1.8 and PCL v 1.9. Could anybody help me with this problem?

[stale[bot]]

Marking this as stale due to 30 days of inactivity. It will be closed in 7 days if no further activity occurs.

[kunaltyagi]

To summarize this, and possibly generate some interest:

1. adding AABB

PR welcome

2. adding grid

4152 exposes some more functionality.

3. using scene graph inside the PCL visualizer
4. event handler enhancement for each object added into scene graph in PCL visualizer

Inherit from visualizer

5. I do really need some examples, detailed documentation

Good point for GSoD contributions

Effort needed for AABB and piecemeal documentation is not a lot, but does require familiarity with codebase.

Adding relevant labels to generate the list later.