Memory leak when we use "vis.CreateVisualizerWindow" and "vis.CaptureDepthFloatBuffer"

[seodongmin-kor]

Checklist

• [x] I have searched for similar issues.
• [x] For Python issues, I have tested with the latest development wheel.
• [x] I have checked the release documentation and the latest documentation (for main branch).

My Question

Hello. I tested below code and found that there was a memory leak when we use "vis.CreateVisualizerWindow" and "vis.CaptureDepthFloatBuffer"

I really don't know why.... please help me.

#include "open3d/Open3D.h"

int main() {

    open3d::geometry::TriangleMesh mesh;
    std::string filePath = getExecutablePath() + "/data/3wayvv.stl";
    open3d::io::ReadTriangleMesh(filePath, mesh);

    while (1) {
        auto cam = open3d::camera::PinholeCameraParameters();
        cam.intrinsic_ = open3d::camera::PinholeCameraIntrinsic(1280, 720, 925.645203, 923.146912, 639.5, 359.5);
        cam.extrinsic_ = Eigen::MatrixXd::Identity(4, 4);

        open3d::geometry::TriangleMesh mesh = mesh;
        Eigen::MatrixXd zz(4, 4);
        zz << -0.169771, 0.947726, -0.270174, 20.9914,
            -0.106152, -0.290145, -0.951077, 57.288,
            -0.97975, -0.132786, 0.149861, 679.271,
            0, 0, 0, 1;
        mesh.Transform(zz);

        auto ptr = std::make_shared<open3d::geometry::TriangleMesh>(mesh);
        auto vis = open3d::visualization::Visualizer();
        vis.CreateVisualizerWindow("open3d", 1280, 720, 0, 0, false); // memory leak
        vis.AddGeometry({ ptr });
        vis.GetViewControl().SetConstantZNear(0);
        vis.GetViewControl().SetConstantZFar(5000);
        vis.GetViewControl().ConvertFromPinholeCameraParameters(cam);
        vis.PollEvents();
        vis.UpdateRender();
        auto depth = vis.CaptureDepthFloatBuffer(false); // memory leak
    }
}

[seodongmin-kor]

And I saw " CreateVisualizerWindow() " description on c++ documnet. There is a comment like this. "This function MUST be called from the main thread." What is that mean?

[hernot]

That means 1) in General you must write your program such that there exists exactly one thread which handles all the gui manipulations and calls all the methods of the visualizer classes. Any other thread must call post_to_main_thread passing a function pointer to the function executing all the required manipulations. 2) That in most programs the main thread (established by your os) is likely to handle all the gui stuff 3) That issue #3914 has very low priority otherwise it would definitively only matter that there exists one dedicated thread handling all the gui and rendering stuff, but not which one. At the moment it still matters when your program ends. Cause at that time all threads exempt the main thread will be properly shutdown already and if you did not dedicate the main thread to handling all the gui and rendering fillament library as not decomissioned until then will throw you an exception about decomissioning beeing executed from the wrong thread, eventhough there is no other. With suggestions from issiue #3914 implemented this exception would only be raised in case of uncontrolled exit or when you as a programmer would not explicitly take care that filament library is decomissioned before your dedicated gui thread stops.

[seodongmin-kor]

thanks!

[seodongmin-kor]

Hello. As you suggest above, I modify and simplify my code for accurate debugging.

#include "open3d/Open3D.h"

int main() {

    // virtual camera setting -------------------------------------------------------------------------------------------------
    auto cam_vir = open3d::camera::PinholeCameraParameters();
    cam_vir.intrinsic_ = open3d::camera::PinholeCameraIntrinsic(720, 540, 1075.65091572, 1073.90347929, 359.5, 269.5);
    cam_vir.extrinsic_ = Eigen::MatrixXd::Identity(4, 4);

    auto vis = open3d::visualization::Visualizer();
    vis.CreateVisualizerWindow("open3d", 720, 540, 0, 0, false); // This function MUST be called from the main thread
    vis.GetViewControl().SetConstantZNear(0);
    vis.GetViewControl().SetConstantZFar(5000);
    // ------------------------------------------------------------------------------------------------------------------------

    while (1) {
        std::cout << "hello ~~" << std::endl;
    }
    return 0;
}

I set termination points within while(1) and detect memory leaks through the Visual Studio Hip Profiling tool.

As shown in the picture below, I've seen intermittent memory leaks in ntdll.dll. If I don't use the open3d::visualization::Visualizer() feature, it won't leak, isn't this an open3d bug? (OS : Window 10 Pro, IDE : Visual Studio 2022, SDK : Open3d 0.17.0)

[overview]

[snapshot 7]

[snapshot 8]

[snapshot 9]