Depth sensing not working on Samsung S20+ 5G?

[jomijomi]

Chrome/Android updated to latest, WebXR flag enabled, but AR Depth Sensing samples (Proposals) doesn't give me anything. Same with Chrome's samples from: https://www.chromestatus.com/feature/5742647199137792#details i.e. https://storage.googleapis.com/chromium-webxr-test/latest.html?target=proposals/phone-ar-depth-gpu.html Chrome's samples are slightly modified from those found here and give me "Depth data unavailable in the current frame!"

Mainly asking here in order to get someone to confirm this is a S20+ problem before I start digging deeper... Anyone else?

Best, Mikael

[bialpio]

Can you share the version number of Chrome that you're using (you can check it by navigating to chrome://version)? The API underwent some amount of changes so depending on the version of Chrome, the samples found in immersive-web may not work. The "Depth data unavailable in the current frame!" message that you're seeing is what the sample displays when it is correctly getting poses from WebXR API on a frame, but the depth data is not being returned. Can you also check if there are any errors logged to the console (guide)?

Internally, Chrome uses ARCore for its AR capabilities - a quick way to check if depth API is supposed to work would be to try the AR Depth Lab on your device. It may also be that what you're seeing is a bug in Chrome. I've opened a new issue to track this.

[jomijomi]

Chrome version: 89.0.4389.105, but I also see same behaviour in Beta and Dev. Depth API works in AR Depth Lab. Haven't been able to do any remote debugging as this is a (brand new) company phone and Developer options are "locked" by default. Sending angry emails to IT-support as we speak to get this unlocked... Best, Mikael

[jomijomi]

Haven't been able to work on this, but I just saw that in the AR Depth Lab in the "Collider" sample, real world objects no longer occlude virtual objects. This used to work previously (when I posted the issue in the first place). Everything updated to latest (firmware, etc.). The "Depth Map" sample appears to be working. A bit laggy, though (ghosting). /Mikael

[stspanho]

Hi, it's also not working on my s20+. The error I'm getting is "Depth data unavailable in the current frame". The AR depth lab app works.

[HyroVitalyProtago]

I think it is not related to your phones, seems to be just some changes in the api names, you can check https://github.com/HyroVitalyProtago/webxr-samples/commit/e66da2ef31e5fc483fa7e3cdf528469334f05d03 and directly try on https://webxr-depth.glitch.me/ (also remember to enable the flag on chrome android #webxr-incubations). PS: there is some transforms errors, but at least you can see something.

[bialpio]

There should be no need to enable #webxr-incubations flag to have the depth API working. I have tried both the CPU and GPU samples on a Pixel 3 & Pixel 6 and did not find any issues with those.

The "Depth data unavailable" message should be displayed by the site when there was no data returned from the device side to Blink (note: all this is Chrome-specific). One way to diagnose would be to look at the trace captured during the repro of the issue. Steps are described here, only xr and xr.debug categories are relevant (it'd be ideal to trace the entire XR session, so start the trace before entering AR). When you have a trace, you can open it and look for "Depth buffer resolution (in pixels)" entries - if there are none (but other XR-related entries are present), it'd point to some issue in Chrome. If the entries are present and width * height is higher than 43200, then it's working as intended - due to privacy reasons, we don't hand out data that has too much detail and we currently do not downsample it.

[HyroVitalyProtago]

The WebXR samples on https://storage.googleapis.com/chromium-webxr-test/r914571/proposals/index.html works for me (on samsung S8) but not those on https://immersive-web.github.io/webxr-samples/proposals/

[stspanho]

@bialpio I've done a trace. I selected the XR category for my trace? There are records present.

For example:

{"args":{},"cat":"xr","dur":1,"name":"ProcessAnchorsData","ph":"X","pid":3342,"tdur":2,"tid":3359,"ts":160235316714,"tts":10393391},
{"args":{"value":100},"cat":"xr","name":"XR viewport scale (%)","ph":"C","pid":3342,"tid":3359,"ts":160227103335,"tts":9522304},
{"args":{"value":100},"cat":"xr","name":"XR viewport scale (%)","ph":"C","pid":3342,"tid":3359,"ts":160227140387,"tts":9526188},

But no "Depth Buffer resolution" entries.

Tested on Chrome (99.0.4844.73) and Samsung Galaxy S20+ 5G. Any idea?

[jomijomi]

Yepp, I still can't get this to work on my Samsung S20+ 5G (Getting the "Depth data unavailable in the current frame!"). However, the same samples work on my other phone, the Samsung Galaxy Note20 5G. Both phones updated to latest on everything. /Mikael

[bialpio]

@stspanho, just to confirm, did you also enable the "xr.debug" category? It would be listed in the "Disabled by Default Categories" list when recording a trace.

[stspanho]

@bialpio idd, I wasn't aware of the 'disabled by default' category. Now the records are present, but still receiving the same error. What could be the problem?

{"args":{"value":33},"cat":"disabled-by-default-xr.debug","name":"ARCore camera frame interval (ms)","ph":"C","pid":15437,"tid":14302,"ts":12888208352,"tts":245308},
{"args":{"height":360,"width":640},"cat":"disabled-by-default-xr.debug","name":"Depth buffer resolution (in pixels)","ph":"C","pid":15437,"tid":14302,"ts":12888217913,"tts":246287},
{"args":{"value":100},"cat":"xr","name":"XR viewport scale (%)","ph":"C","pid":13514,"tid":13530,"ts":12888223067,"tts":3813559},

[bialpio]

Thanks, this is exactly what we need!

So it looks like we're getting a 360x640 buffer. The "problem" is that our implementation in Chrome artificially suppresses handing out depth information if the buffer resolution is too high. This is in response to privacy concerns - returning depth buffers with high resolution could expose too fine-grained data to the site, and we don't implement downscaling the data yet. :(

[stspanho]

@bialpio ok I understand. Not much I can do about it at the moment. Is the downscaling on the roadmap?

[guo1017138]

@bialpio I have the same issue as this thread when reading depth data and null returned with ToF camera. But what I meant is to get high resolution depth data and don't want to downscaling it. So I am wondering is there a way to read the high-resolution data directly? Or how to set the privacy to make chrome allow high-resolution data? I am getting blocked here.

Thanks