search

HengyiWang / Co-SLAM

[CVPR'23] Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM
https://hengyiwang.github.io/projects/CoSLAM.html
Apache License 2.0
399 stars 35 forks source link

When to release data capture by ipad / iphone? #6

Open YiChenCityU opened 1 year ago

YiChenCityU commented 1 year ago

Thanks very much.

HengyiWang commented 1 year ago

Thank you for your interest, I will try to release the tutorials and the related code for capturing data using ipad/iphone by the end of this week.

wing-kit commented 1 year ago

Thank you for sharing the way to prepare dataset using iPhone/iPad LiDAR. I am able to run the steps and get the reconstructed mesh using iPad Pro.

mesh

BTW Do you have an idea how to get a more photorealistic result?

I also tried the apartment dataset from nice-slam (captured from azure kinect dk). The appearance of the reconstructed mesh is also not promising, given that the azure one has much higher depth map resolution.

HengyiWang commented 1 year ago

Hi @wing-kit, it is great to see that you manage to capture data using iPhone/iPad and get the reconstruction results using Co-SLAM.

Regarding your question about obtaining more photorealistic results, I have some suggestions for you to try:

  1. In the latest commit of Co-SLAM, I added an option for rendering surface color. You can give it a try and see if it improves the photorealism of the reconstructed mesh. However, please keep in mind that the viewing direction is currently set to be the surface normal. If the actual viewing direction significantly deviates from the surface normal, the color might appear worse.

  2. Co-SLAM performs joint optimization of both geometry and color. To enhance the color quality, you can experiment with increasing the weight of the RGB loss or increasing the number of mapping iterations (iters: 10). Note that the current hyper-parameter setting focuses on computational efficiency and uses only 5% of pixels (n_pixels: 0.05) from each keyframe. You can consider increasing this percentage to obtain more pixel observations

  3. By default, the voxel size for mesh extraction in Co-SLAM is around 3cm, which might be slightly large for small objects. Reducing the voxel size further may help improve the small objects in the reconstructed mesh.

I hope these suggestions help you in achieving more photorealistic results with your reconstructed meshes. Let me know if you have any further questions or need assistance with anything else:)