lessthanoptimal / BoofCV

Fast computer vision library for SFM, calibration, fiducials, tracking, image processing, and more.
http://boofcv.org
1.09k stars 261 forks source link

Run sfm examples, save the 3d model in ply format but it does not display correctly in other tools(meshlab, 3d viewer,...). #232

Open ApplicateIT-Thang opened 3 years ago

ApplicateIT-Thang commented 3 years ago

Dear @lessthanoptimal and team, I am very interested in the sfm module of BoofCV. I tried to run the examples module from the command: java -jar examples/examples.jar , and run the ExampleMultiViewDenseReconstruction.

After finish processing, the 3d model was reconstructed and visualized very well in PointCloudViewer.

pointcloudviewer_result

Then I tried to save the 3d model in ply format by following code: _// Save to the 3d ply model try { DogArray savedCloud = viewer.copyCloud(null); var workDirectory = "mvs_work/"; OutputStream outputStream = new FileOutputStream(new File(workDirectory, "pointcloud.ply")); PointCloudIO.save3D(PointCloudIO.Format.PLY, PointCloudReader.wrapF64RGB(savedCloud.toList()), false, outputStream); outputStream.close(); } catch (Exception e) { e.printStackTrace(); }

The 3d model is generated. But when I open this model by tool such as meshlab or 3d viewer online, the model displayed incorrectly, not as same as the PointCloudViewer showed.

save_to_ply_then_display_in_3d_viewer_tool

Did I do something wrong when saving the 3d model? Or how can I save the 3d model which other tools are able to display the correct model like the PointCloudViewer?

Thank you very much for making this great library. Regards. KieuThang

lessthanoptimal commented 3 years ago

I ran into the same problem too. What's going on is the point cloud has unfiltered stereo noise. The noise will throw some points out towards infinity. That's bad when your program tries to be smart and auto scale based on the point distribution! Clearly a bit of work is going to be needed to make this play nice with other tools. That will probably happen when meshing gets added.

I've had some success using osgviewer in Ubuntu. It would initially look like what you showed above and eventually I could move the camera much closer to where it's in the BoofCV viewer.

Loop closure for sure is coming in the next release. It might be a bit longer until the point cloud to 3D mesh is added and much of this noise is filtered. Unless someone wants to help out with that!

On Thu, Mar 11, 2021 at 7:41 PM ApplicateIT-Thang @.***> wrote:

Dear @lessthanoptimal https://github.com/lessthanoptimal and team, I am very interested in the sfm module of BoofCV. I tried to run the examples module from the command: java -jar examples/examples.jar , and run the ExampleMultiViewDenseReconstruction .

After finish processing, the 3d model was reconstructed and visualized very well in PointCloudViewer. [image: pointcloudviewer_result] https://user-images.githubusercontent.com/73826280/110888595-108f8c80-831f-11eb-96e7-545f1ff52d55.png

Then I tried to save the 3d model in ply format by following code:

// Save to the 3d ply model try { DogArray savedCloud = viewer.copyCloud(null); var workDirectory = "mvs_work/"; OutputStream outputStream = new FileOutputStream(new File(workDirectory, "point_cloud.ply")); PointCloudIO.save3D(PointCloudIO.Format.PLY, PointCloudReader.wrapF64RGB(savedCloud.toList()), false, outputStream); outputStream.close(); } catch (Exception e) { e.printStackTrace(); }

The 3d model is generated. But when I open this model by tool such as meshlab or 3d viewer online, the model displayed incorrectly, not as same as the PointCloudViewer showed. [image: save_to_ply_then_display_in_3d_viewer_tool] https://user-images.githubusercontent.com/73826280/110888624-21d89900-831f-11eb-8996-4409e0f1e685.png

Did I do something wrong when saving the 3d model? Or how can I save the 3d model which other tools are able to display the correct model like the PointCloudViewer?

Thank you very much for making this great library. Regards. KieuThang

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lessthanoptimal/BoofCV/issues/232, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFUOV7FKKV4U6PMCLS2U5TTDGEO7ANCNFSM4ZBP6TFA .

-- "Now, now my good man, this is no time for making enemies." — Voltaire (1694-1778), on his deathbed in response to a priest asking that he renounce Satan.

ApplicateIT-Thang commented 3 years ago

Hi, Thank you very much for your helpful information, I am reading 5-Step Guide to generate 3D meshes from point clouds with Python, but it's written from Python.

Hope it helps you a bit in this case. Regards.

lessthanoptimal commented 3 years ago

Small update on this. The latest example code will save and prune "far" points. This makes viewing in 3rd party apps much easier.

lessthanoptimal commented 3 years ago

image

ApplicateIT-Thang commented 3 years ago

Hi @lessthanoptimal, Thank you for your update. But I still get the issue when opening the saved model on another viewer.

Screen Shot 2021-06-17 at 11 05 43

The model is not clear and could not be viewed.

lessthanoptimal commented 3 years ago

Looks like that viewer is incorrectly reading the PLY file and treating it like a set of triangles. Meshing won't make it into the up coming release, which is being finalized now, but should in the next one.

lessthanoptimal commented 2 years ago

In BoofCV 0.41 there is now a way to save disparity and inverse depth images as a 3D Mesh. These tend to work well in 3rd party viewers. There is also a plan in place for how to convert general 3D clouds into 3D meshes. Work might start on that in the next release.

https://boofcv.org/index.php?title=Example_Stereo_Mesh