How can i save the final model as `.off` format ?

[Wjiajie]

I add the code:

//save pointcloud
saveCorrespondedCloud(observation,fused_geometry_idx,save_dir);

in order to save the final model as point cloud format , But i get Segmentation fault error when runing. The debug logs :

1  Eigen::internal::copy_using_evaluator_DefaultTraversal_CompleteUnrolling<Eigen::internal::generic_dense_assignment_kernel<Eigen::internal::evaluator<Eigen::Block<Eigen::Matrix<float, 3, -1, 0, 3, -1>, 3, 1, true>>, Eigen::internal::evaluator<Eigen::Block<Eigen::Matrix<float, 4, 1, 0, 4, 1>, 3, 1, false>>, Eigen::internal::assign_op<float, float>, 0>, 0, 3>::run                                                                                                                                                                                                                                                                                                   AssignEvaluator.h          207 0x5555556d7093 
2  Eigen::internal::dense_assignment_loop<Eigen::internal::generic_dense_assignment_kernel<Eigen::internal::evaluator<Eigen::Block<Eigen::Matrix<float, 3, -1, 0, 3, -1>, 3, 1, true>>, Eigen::internal::evaluator<Eigen::Block<Eigen::Matrix<float, 4, 1, 0, 4, 1>, 3, 1, false>>, Eigen::internal::assign_op<float, float>, 0>, 3, 2>::run                                                                                                                                                                                                                                                                                                                                     AssignEvaluator.h          435 0x5555556d7093 
3  Eigen::internal::call_dense_assignment_loop<Eigen::Block<Eigen::Matrix<float, 3, -1, 0, 3, -1>, 3, 1, true>, Eigen::Block<Eigen::Matrix<float, 4, 1, 0, 4, 1>, 3, 1, false>, Eigen::internal::assign_op<float, float>>                                                                                                                                                                                                                                                                                                                                                                                                                                                        AssignEvaluator.h          741 0x5555556d7093 
4  Eigen::internal::Assignment<Eigen::Block<Eigen::Matrix<float, 3, -1, 0, 3, -1>, 3, 1, true>, Eigen::Block<Eigen::Matrix<float, 4, 1, 0, 4, 1>, 3, 1, false>, Eigen::internal::assign_op<float, float>, Eigen::internal::Dense2Dense, void>::run                                                                                                                                                                                                                                                                                                                                                                                                                               AssignEvaluator.h          879 0x5555556d7093 
5  Eigen::internal::call_assignment_no_alias<Eigen::Block<Eigen::Matrix<float, 3, -1, 0, 3, -1>, 3, 1, true>, Eigen::Block<Eigen::Matrix<float, 4, 1, 0, 4, 1>, 3, 1, false>, Eigen::internal::assign_op<float, float>>                                                                                                                                                                                                                                                                                                                                                                                                                                                          AssignEvaluator.h          836 0x5555556d7093 
6  Eigen::internal::call_assignment<Eigen::Block<Eigen::Matrix<float, 3, -1, 0, 3, -1>, 3, 1, true>, Eigen::Block<Eigen::Matrix<float, 4, 1, 0, 4, 1>, 3, 1, false>, Eigen::internal::assign_op<float, float>>(Eigen::Block<Eigen::Matrix<float, 3, -1, 0, 3, -1>, 3, 1, true>&, Eigen::Block<Eigen::Matrix<float, 4, 1, 0, 4, 1>, 3, 1, false> const&, Eigen::internal::assign_op<float, float> const&, Eigen::internal::enable_if<!Eigen::internal::evaluator_assume_aliasing<Eigen::Block<Eigen::Matrix<float, 4, 1, 0, 4, 1>, 3, 1, false>, Eigen::internal::evaluator_traits<Eigen::Block<Eigen::Matrix<float, 4, 1, 0, 4, 1>, 3, 1, false>>::Shape>::value, void *>::type) AssignEvaluator.h          804 0x5555556d7093 
7  Eigen::internal::call_assignment<Eigen::Block<Eigen::Matrix<float, 3, -1, 0, 3, -1>, 3, 1, true>, Eigen::Block<Eigen::Matrix<float, 4, 1, 0, 4, 1>, 3, 1, false>>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             AssignEvaluator.h          782 0x5555556d7093 
8  Eigen::MatrixBase<Eigen::Block<Eigen::Matrix<float, 3, -1, 0, 3, -1>, 3, 1, true>>::operator=<Eigen::Block<Eigen::Matrix<float, 4, 1, 0, 4, 1>, 3, 1, false>>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 Assign.h                   66  0x5555556d7093 
9  transformPointCloud                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           common_point_cloud_utils.h 97  0x5555556d7093 
10 surfelwarp::Visualizer::SaveMatchedCloudPair                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  PointCloudVisualizer.cpp   433 0x5555556d7093 
11 surfelwarp::Visualizer::SaveMatchedCloudPair                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  PointCloudVisualizer.cpp   446 0x5555556d7093 
12 surfelwarp::Visualizer::SaveMatchedCloudPair                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  PointCloudVisualizer.cpp   461 0x5555556d7093 
13 surfelwarp::SurfelWarpSerial::saveCorrespondedCloud                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           SurfelWarpSerial.cpp       323 0x55555561a5d1 
14 surfelwarp::SurfelWarpSerial::ProcessNextFrameWithReinit                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      SurfelWarpSerial.cpp       284 0x55555561d3df 
15 main                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          main.cpp                   41  0x5555555fb980 

It seems some error occur in:

Visualizer::SaveMatchedCloudPair(
        observation.vertex_config_map,
        geometry.live_vertex_confid.ArrayView(),
        m_camera.GetCamera2WorldEigen(),
        cloud_1_name, cloud_2_name
    );

inside the fuction saveCorrespondedCloud(observation,fused_geometry_idx,save_dir);.

Am I passing the wrong paras ? Could you help me out to see what's wrong?Thank.

[weigao95]

Can you make sure geometry.live_vertex_confid.ArrayView() has correct size? You can get it by geometry.live_vertex_confid.ArrayView().Size().

[Wjiajie]

Sory for the late reply~ The result is:

/home/jiajie/body-reconstruction/surfelwarp/apps/surfelwarp_app/main.cpp:40: The 0th Frame
geometry.live_vertex_confid.ArrayView().Size():76728
/home/jiajie/body-reconstruction/surfelwarp/apps/surfelwarp_app/main.cpp:40: The 1th Frame
geometry.live_vertex_confid.ArrayView().Size():76904
/home/jiajie/body-reconstruction/surfelwarp/apps/surfelwarp_app/main.cpp:40: The 2th Frame
geometry.live_vertex_confid.ArrayView().Size():77042

BTW,the subfuction :

//Also save the reference point cloud
    Visualizer::SavePointCloud(geometry.reference_vertex_confid.ArrayView(), (save_dir / "reference.off").string());

inside the fuction saveCorrespondedCloud(observation,fused_geometry_idx,save_dir); works well, But somethings wrong with :

Visualizer::SaveMatchedCloudPair(
        observation.vertex_config_map,
        geometry.live_vertex_confid.ArrayView(),
        m_camera.GetCamera2WorldEigen(),
        cloud_1_name, cloud_2_name
    );

[Wjiajie]

When i call:

void surfelwarp::Visualizer::SaveMatchedCloudPair(
        cudaTextureObject_t cloud_1,
        const DeviceArray<float4> &cloud_2,
        const Eigen::Matrix4f &from1To2,
        const std::string &cloud_1_name, const std::string &cloud_2_name
) {
    const auto h_cloud_1 = downloadPointCloud(cloud_1);
    const auto h_cloud_2 = downloadPointCloud(cloud_2);
    SaveMatchedCloudPair(
            h_cloud_1,
            h_cloud_2,
            from1To2,
            cloud_1_name, cloud_2_name
    );
}

h_cloud_1 return all zero value in h_cloud_1.points ,while h_cloud_2 return reasonable value. For example:

    Locals      
        cloud_1 <optimized out> 
        cloud_1_name    "frame_1/observation.off"   std::__cxx11::string &
        cloud_2 @0x7fffffffd9a0 surfelwarp::DeviceArray &
        cloud_2_name    "frame_1/model.off" std::__cxx11::string &
        from1To2    (4 x 4), ColumnMajor    Eigen::Matrix4f &
        h_cloud_1   @0x555570001660 std::shared_ptr<cilantro::PointCloud<float, 3> >
            colors  <not accessible>    Eigen::Matrix<float, 3, -1, 0, 3, -1>
            normals <not accessible>    Eigen::Matrix<float, 3, -1, 0, 3, -1>
            points  (3 x 264000), ColumnMajor   cilantro::VectorSet
                [0,0]   -0.0    float
                [1,0]   -0.0    float
                [2,0]   0.0 float
                [0,1]   -0.0    float
                [1,1]   -0.0    float
                [2,1]   0.0 float
                [0,2]   -0.0    float
                [1,2]   -0.0    float
                [2,2]   0.0 float
                [0,3]   -0.0    float
                [1,3]   -0.0    float
                [2,3]   0.0 float
                    ......          
        h_cloud_1@1 <optimized out> 
        h_cloud_2   @0x55555dab58d0 std::shared_ptr<cilantro::PointCloud<float, 3> >
            colors  <not accessible>    Eigen::Matrix<float, 3, -1, 0, 3, -1>
            normals <not accessible>    Eigen::Matrix<float, 3, -1, 0, 3, -1>
            points  (3 x 76728), ColumnMajor    cilantro::VectorSet
                [0,0]   290.2420959472656   float
                [1,0]   -174.35791015625    float
                [2,0]   606.0   float
                [0,1]   291.3052673339844   float
                [1,1]   -174.35791015625    float
                [2,1]   606.0   float
                [0,2]   292.3684387207031   float
                [1,2]   -174.35791015625    float
                [2,2]   606.0   float
                [0,3]   290.2420959472656   float
                [1,3]   -173.29473876953125 float
                    ......

Is it something wrong occour when return the value from GPU device to host ?

[weigao95]

I guess you add the save command to a place where sync has not finished. Where did you add the code? Can you attach your file?

[Wjiajie]

I add the code in void surfelwarp::SurfelWarpSerial::ProcessNextFrameWithReinit(bool offline_save) of SurfelWarpSerial.cpp:

//Debug save
    if(offline_save) {
        const auto with_recent = draw_recent || use_reinit;
        const auto& save_dir = createOrGetDataDirectory(m_frame_idx);
        saveCameraObservations(observation, save_dir);
        saveSolverMaps(solver_maps, save_dir);

        const auto num_fused_vertex = m_surfel_geometry[fused_geometry_idx]->NumValidSurfels();
        saveVisualizationMaps(
            num_fused_vertex, fused_geometry_idx,
            m_camera.GetWorld2CameraEigen(), m_camera.GetInitWorld2CameraEigen(),
            save_dir, with_recent
        );

        //Here the place i add the code
        saveCorrespondedCloud(observation,fused_geometry_idx,save_dir);
    }

I did not chage any code of SurfelWarpSerial.cpp except here.

[weigao95]

Can you add a line Visualizer::DrawPointCloud(observation.vertex_config_map) before saveCorrespondedCloud(observation,fused_geometry_idx,save_dir) to make sure the image processing is correct?

[Wjiajie]

I add it ,and show nothing~Whether or not i Comment saveCorrespondedCloud(observation,fused_geometry_idx,save_dir); out

[weigao95]

It means the image processing is not correct as there is not valid vertex. I would recommend to debug the image processing using the surfelwarp/apps/imgproc_app/main.cpp application to make sure the rgbd image, vertex and normal map are correct. Also make sure the far-away clipping value is reasonable.

[Wjiajie]

Thanks! I will work it out myselves~

[wine3603]

@Wjiajie hi, I got the same problem. I want to reconstruct the mesh in form of obj or ply, @weigao95 Could you guild me how to generate the mesh from the generated *.png files?

[Wangyouai]

Thanks! I will work it out myselves~

I can also use this functionVisualizer::SavePointCloudto save the point cloud，But can't usesaveCorrespondedCloud It reported the following error on clion:

/home/wya/project/SW/surfelwarp/apps/surfelwarp_app/main.cpp:86: The 0th Frame
surfelwarp_app: /usr/local/include/eigen3/Eigen/src/Core/DenseCoeffsBase.h:365: Eigen::DenseCoeffsBase<Derived, 1>::Scalar& Eigen::DenseCoeffsBase<Derived, 1>::operator()(Eigen::Index, Eigen::Index) [with Derived = Eigen::Matrix<float, 3, -1>; Eigen::DenseCoeffsBase<Derived, 1>::Scalar = float; Eigen::Index = long int]: Assertion `row >= 0 && row < rows() && col >= 0 && col < cols()' failed.

[BaldrLector]

@Wjiajie Hi, can you ran Visualizer::SaveMatchedCloudPair successfully? In my case, I can run Visualizer::SavePointCloud and the Visualizer::DrawPointCloud, but Visualizer::SaveMatchedCloudPair return malloc error.

[yuehaowang]

I used to fail in running Visualizer::SaveMatchedCloudPair as well. But after I switched to building with PCL, the problem was gone. Perhaps this problem stems from cilantro. BTW, I also found PCL works on my Ubuntu 18 with CUDA 11.2.

[shaisung]

I used to fail in running Visualizer::SaveMatchedCloudPair as well. But after I switched to building with PCL, the problem was gone. Perhaps this problem stems from cilantro. BTW, I also found PCL works on my Ubuntu 18 with CUDA 11.2.

I've had similar experience (probably unrelated to original issue in this thread though). I think it has to do with custom memory aligned allocators used in Eigen (which is used by cilantro). Not 100% sure.

As far as I can tell, it was solved by using make_shared instead of (new T) in data_transfer.cu, e.g.:

PointCloud3f_Pointer point_cloud = std::make_shared<PointCloud3f>(); //(new PointCloud3f);

There was also some mis-use of input->points.size() instead of input.size().