search

googlesamples / arcore-depth-lab

ARCore Depth Lab is a set of Depth API samples that provides assets using depth for advanced geometry-aware features in AR interaction and rendering. (UIST 2020)
https://augmentedperception.github.io/depthlab/
Apache License 2.0
762 stars 152 forks source link

Implemetation of ARCore Depth API functions in AR Foundation 4.x.x? #20

Closed RamoramaInteractive closed 3 years ago

RamoramaInteractive commented 3 years ago

I want to implement some depth functions of the ARCore Depth API into an already existing AR Foundation project. Is it possible to copy the scripts and adapt the gradle settings?

I want to improve the functionalities to toss a dice on a ground with occlusions. You should be able to cover objects with your hand or a person.

ruofeidu commented 3 years ago

I played with latest AR Foundation last month and it supports Depth API. You may have to tweak the depth input APIs in DepthSource.cs to get the correct input of Depth. The core idea is: DepthLab only needs depth map, intrinsic and extrinsic matrices of the camera, and RGB image to work with. See https://augmentedperception.github.io/depthlab/ for more detail.

RamoramaInteractive commented 3 years ago

How can I tweak the depth input in DepthSource.cs for AR Foundation? Are there any examples how to adapt the depth api for AR Foundation?

ruofeidu commented 3 years ago

Sorry but we don't own the code of ARFoundation but it is possible to tweak the code to make DepthLab compatible with ARFoundation. Or can we use ARFoundation and ARCore SDK at the same time? I am not very familiar with ARFoundation.

ericloureiro commented 3 years ago

...Or can we use ARFoundation and ARCore SDK at the same time?.

Short answer: No. We can not use both at the same time in Unity. Can not pass EnsureGoogleARCoreIsNotPresent() validation: Error building Player: BuildFailedException: GoogleARCore detected. Google's "ARCore SDK for Unity" and Unity's "ARCore XR Plugin" package cannot be used together. If you have already removed GoogleARCore, you may need to restart the Editor.

I'm having a problem similar to the OP, but in my case, I want to use Occlusion Transparency from Depth Lab as it is not provided by Occlusion Subsytem from AR Foundation.

In general, I too want to use ARCore Depth API in an AR Foundation project. Is there a way to adapt the scripts? Being trying to tweak here and there on the API code but without so much luck.

ruofeidu commented 3 years ago

We recently released new updates using AR Foundation: https://github.com/googlesamples/arcore-depth-lab/releases please feel free to check it out :)