search

googlesamples / arcore-depth-lab

ARCore Depth Lab is a set of Depth API samples that provides assets using depth for advanced geometry-aware features in AR interaction and rendering. (UIST 2020)
https://augmentedperception.github.io/depthlab/
Apache License 2.0
780 stars 153 forks source link

Continuous evironment mesh building #34

Closed Ademord closed 1 year ago

Ademord commented 3 years ago

hello @ruofeidu, thank you for all the effort you guys put into making this repo and the research behind it. i am doing something similar to #31, and I am looking at KinectFusion and some SLAM alternative.

One question though, how could I reuse the primitives that you have for obtaining the depth information and the point cloud? I have found resources online that i have to write my own shaders etc, and was wondering if there as an API I could use (I have not tried or used the ARCore Raw Depth API, could you maybe explain what is possible and what is not possible with it? what did you have to do on your own that I could maybe reuse from your work?

ruofeidu commented 3 years ago

Hi Franco,

Sorry that I'm out of office this week on vacation; I'll try to revisit next Tuesday. There is a raw point cloud scene with main API and script in 1) (Using AR Foundation) https://github.com/googlesamples/arcore-depth-lab/blob/master/Assets/ARRealismDemos/PointCloud/Scripts/RawPointCloudBlender.cs 2) https://github.com/googlesamples/arcore-depth-lab/blob/arcore_unity_sdk/Assets/ARRealismDemos/PointCloud/Scripts/RawPointCloudBlender.cs

Only very basic algorithm is implemented, and I look forward to real-time KinectFusion with it :D

Cheers, Ruofei

Ademord commented 3 years ago

Hi @ruofeidu and thank you for your feedback! I posted in the unity forums to figure out how to use a normal game camera with the ARkit.. I assume its made for only iphones ? And i have no idea how to access the depth textures as of today from a normal game cam. Enjoy your holidays!!

ruofeidu commented 3 years ago

Thank you! BTW, this repo is only targeted at Android and ARCore :) Not for iPhone as stated in the README.md

Darkar25 commented 2 years ago

i see this repo is quite abandoned but i still hope this feature will be made by someone in somewhat near future

ruofeidu commented 2 years ago

Hi Darkar25,

I won't say it's abandoned - all algorithms are still working in the legacy arcore-unity-sdk branch and it's fairly straight forward to port them in other forms. For example, our recent launch of 3D portrait photo https://blog.tensorflow.org/2022/05/portrait-depth-api-turning-single-image.html?linkId=8063793 is rewrite of DepthLab 3D photo scene.

Cheers, Ruofei