Focus control for handheld AR

[klausw]

For smartphone-based AR, the underlying device may support a choice of focus modes. For example, ARCore has a selectable focus mode with a choice of FIXED or AUTO.

Currently, WebXR doesn't have a way to express such a preference, and this can lead to problems in specific use cases, for example: https://github.com/google/model-viewer/issues/2470

If I'm understanding it right, ARCore's default is a fixed-focus mode since that allows more accurate tracking, for example many camera lenses have a "focus breathing" effect where the field of view changes slightly as focus is being adjusted. However, this isn't a good fit for some scenarios such as marker tracking close to the camera.

Would it make sense to add a way for apps to express their preferred focus mode, and/or making it configurable at runtime? This also seems applicable for AR headsets or other use cases that may be using a camera view with a choice of focus modes.

There's a few options here:

• let devices make their own choice with no user override, for example always enabling autofocus even if that degrades some use cases, or vice versa for fixed focus

• add XRSessionInit features with "fixed-focus" and "auto-focus" options

• add a XRSession setCameraFocusMode(x) method, with enum values fixed-far, fixed-near, auto or similar? This would be a hint to the runtime where it would choose the most appropriate corresponding supported value.

• use the depthNear and depthFar attribute to infer at which range the app expects to be doing interactions? This seems a bit too magical since applications typically just pick fairly arbitrary values for these.

• automatically turn on autofocus if features are in use that are likely to benefit from it, for example marker-tracking? This could have the unintended side effect that people may turn on a feature just to change the focus mode if this appeared to help in their specific use case.

• more complicated options such as a per-camera setting for devices with multiple cameras?

Do people have preferences in this area, and would it also apply to non-AR use cases involving cameras?

[bialpio]

/tpac to touch base on this proposal

[bialpio]

Summary: discussed this at TPAC, the group is leaning towards letting the UA decide (so no new API is needed), either by enabling auto-focus by default or by leveraging other signals to make the decision (e.g. by inspecting the requested/optional features).

[papadako]

Hello,

it seems that autofocus does not work anymore with the image tracking feature in WebXR. Is there any way to enable it?

[Shyiy]

Has auto-focus become available already somehow, maybe as consequence of one of the features available to be passed? I still couldn't find anything regarding this...

[HedwigAR]

Hi,

I'm personally okay with any implementation of auto-focus, just having a way to set it would be a great help. Currently when inspecting small tabletop models with an Android Phone the experience seems of a lower quality which it isn't. I personally haven't observed the 'camera-breathing' effect, though I can imagine that happening.

[hybridherbst]

Bump! Especially marker tracking suffers a lot from the current constantly-out-of-focus experience for users.

[papadako]

Dear all,

can we please have an update on this?

[Manishearth]

@papadako it seems like your issue is with a specific implementation of this standard, which is out of scope for this issue tracker.

As mentioned above, the decision of the group was to have the user agent (i.e. the browser) handle this. If browsers are not handling this correctly, that is a browser bug. In the future we may decide to also add new APIs for it, but that is not currently planned.

[hybridherbst]

@Manishearth has this group decision been captured anywhere? I don't think https://github.com/immersive-web/marker-tracking/blob/main/explainer.md mentions anything about focus and how implementations are supposed to handle it.

[Manishearth]

I don't see how this is super relevant for the marker tracking spec.

In general the WebXR spec and the WebXR AR module do not constrain how the user agent may choose to display the real world environment to the device. So in a sense this group decision is already reflected in the spec. Non normative text to this effect could be added but I do not perceive it as being overly necessary.

[papadako]

@papadako it seems like your issue is with a specific implementation of this standard, which is out of scope for this issue tracker.

As mentioned above, the decision of the group was to have the user agent (i.e. the browser) handle this. If browsers are not handling this correctly, that is a browser bug. In the future we may decide to also add new APIs for it, but that is not currently planned.

Yes, my issue is with Chrome which suddenly stopped using autofocus for the camera. I raised an issue there https://bugs.chromium.org/p/chromium/issues/detail?id=1371883&q=WebXR%20AR&can=1 but it got a WontFix status.

But despite this issue, we should have the freedom to express what we expect from the camera. I think the option to add an XRSession setCameraFocusMode(x) method, with enum values fixed-far, fixed-near, auto or similar will allow each developer to optimize the image tracking performance for his/her specific use case.

[stents]

This issue almost makes WebXR close to unusable for our intended use cases. It's there still no update on this?

We would like to process images obtained via camera-access, however without autofocus the camera feed is a blurry mess up close on most devices.

As suggested, some way of specifying the autofocus mode would make WebXR so much more of a viable option for us.

[MotivaCG]

Hello WebXR Team,

I am writing to refresh the issue we've encountered while using the WebXR image-tracking feature. The primary concern is that the camera does not perform autofocus during AR sessions, which severely affects the usability of this otherwise awesome feature, that becomes useless.

Issue Description: During augmented reality sessions that utilize image tracking, the lack of autofocus results in two main problems:

Image Tracking Accuracy: Without autofocus, the camera often fails to clearly recognize and track the images. This leads to poor tracking performance, as the tracked images are not in clear focus.

AR Mesh Alignment: Meshes added in AR do not align well with the physical environment when the camera is out of focus. This misalignment breaks the immersion and accuracy of augmented reality experiences, making the feature less effective for practical applications.

Expected Behavior: The camera should automatically adjust its focus to ensure that tracked images are sharp and that AR content aligns accurately with the real world. This adjustment should occur continuously during the AR session or at least provide a mechanism to trigger autofocus manually.

Steps to Reproduce:

Initiate an AR session with image tracking enabled. Point the camera at a known target image. Observe that the camera focus does not adjust automatically, resulting in a blurry image and poor tracking performance.

[m0nzderr]

+1. Although the continuous auto-focus idea might be questionable (either due to focus-breathing or variable lens distortion), id would be nice to be able to switch between infinity (current behavior) or close-up modes.

[ryanbarkernz]

+1 Would love some progress on this. My current use case is to simply save close up / in focus images via camera-access that aren't blurry. The main purpose of the app is to take photos and video recordings - but AR lends itself as a guide / helper to the user (eg to take the right photos).

Other than my current use case I've worked on marketing apps in the past that would have wanted this feature for higher quality screen captures for sharing on socials.