[visualization] AreaPickingEvent.getPointsIndices() returns wrong indices

[Ru1yi]

My Environment:

• OS: Windows 11
• Compiler: MSVC2019_64
• PCL Version 1.12.1
• VTK 9.1.0
• Qt 5.15.2

Problem

I integrated the areapickingevent under the visualization module in my Qt program, and tried to get the point cloud index of my selected area through getPointsIndices. But after I visualized the resulting point cloud, I found that the point cloud index was wrong.

The green box in the picture above is the part I selected.

The green box in the above picture is the result of the selected point cloud visualization.

Code

void PCL_test::pp_callback_AreaSelect(const pcl::visualization::AreaPickingEvent& event, void* args)
{
    PCL_test* p = (PCL_test*)args;
    std::vector<int > indices;
    if (event.getPointsIndices(indices) == -1)
        return;

    p->selected_3D_points->clear();

    qDebug() << "Pointcloud area size: " << indices.size() << endl;
    for (int i = 0; i < indices.size(); ++i)
    {
        qDebug() << "Selected point[" << i + 1 << "] indice" << indices[i] << ":" << p->cloudptr->points.at(indices[i]).x << ", " << p->cloudptr->points.at(indices[i]).y << ", " << p->cloudptr->points.at(indices[i]).z << endl;
        p->selected_3D_points->points.push_back(p->cloudptr->points.at(indices[i]));
    }
    //qDebug() << "Select points size: " << p->selected_3D_points->width << "*" << p->selected_3D_points->height << endl;

    //*p->cloudptr = *p->selected_3D_points;
    pcl::visualization::PointCloudColorHandlerCustom<pcl::PointXYZI> cloud_color(p->selected_3D_points, 0, 255, 0);
    p->cloud_viewer->removeShape("pick");
    p->cloud_viewer->addPointCloud(p->selected_3D_points, cloud_color, "pick");
    p->cloud_viewer->setPointCloudRenderingProperties(pcl::visualization::PCL_VISUALIZER_POINT_SIZE, 3, "pick");
    p->ui.qvtkwidget_cloudviewer->update();
}

Additional context I found that getPointsIndices does not have any parameters for viewing angle settings, and I don't know what other factors will affect the accuracy of point cloud selection. In addition, I would like to add that I did not see any specific point cloud selection operation instructions in the official documentation, such as "press x first, then use the left mouse button to select, and then press x to visualize the selection results". I hope the official can add more detailed introductions in subsequent updates. Thanks a lot.

[mvieth]

@internet-person Have you changed all your passwords?

[mvieth]

@Ru1yi Can you please try the most recent PCL version instead, 1.14.1? I believe this issue was fixed since 1.12.1

[Ru1yi]

@Ru1yi Can you please try the most recent PCL version instead, 1.14.1? I believe this issue was fixed since 1.12.1

Do I need to update the versions of MSVC and QT at the same time?

[mvieth]

Do I need to update the versions of MSVC and QT at the same time?

I don't think so. In our CI pipeline, we are still testing Qt 5.12.8, Qt 5.15.3, and Qt 5.15.13 (on Ubuntu). Regarding MSVC I can't say for sure because we are only testing with MSVC 2022, but I don't see a reason why MSVC 2019 shouldn't work as well.

[Ru1yi]

Do I need to update the versions of MSVC and QT at the same time?

I don't think so. In our CI pipeline, we are still testing Qt 5.12.8, Qt 5.15.3, and Qt 5.15.13 (on Ubuntu). Regarding MSVC I can't say for sure because we are only testing with MSVC 2022, but I don't see a reason why MSVC 2019 shouldn't work as well.

I installed PCL1.14.1, updated the include and lib folder locations in the VC++ directory, and also changed the Boost lib names in the linker. The program can run normally. When I press x, I can use the left mouse button to select a box, but the indice size I get is 0， the viewer does not display any selected point cloud.

Here is how I bind AreaPickingEvent, is there anything wrong?

void PCL_test::pushButton_PickArea_Clicked()
{
    HKL hCurKL = NULL;
    // Force switch to English input method
    hCurKL = GetKeyboardLayout(0);
    LoadKeyboardLayout((LPCSTR)QString("0x0409").utf16(), KLF_ACTIVATE);

    QApplication::processEvents();

    if (is_area_selected == false)
    {
        is_area_selected = true;
        ui.pushButton_PickArea->setText("End Area");
        cloud_viewer->registerAreaPickingCallback(pp_callback_AreaSelect, this);
    }
    else
    {
        is_area_selected = false;
        ui.pushButton_PickArea->setText("Pick Area");
        selected_3D_points.reset(new PointCloudT);
    }

[mvieth]

https://github.com/PointCloudLibrary/pcl/blob/master/tools/pcd_viewer.cpp has an example of how to use AreaPickingEvent

[Ru1yi]

https://github.com/PointCloudLibrary/pcl/blob/master/tools/pcd_viewer.cpp has an example of how to use AreaPickingEvent

There is no getCloudNames() method in the AreaPickingEvent class of PCL1.12.1. So I still need to update the PCL version. Thanks a lot anyway. I will try it later.

[mvieth]

@Ru1yi What is the status here? Have you tried with the code from pcd_viewer.cpp?