search

immersive-web / proposals

Initial proposals for future Immersive Web work (see README)
95 stars 11 forks source link

Add support for pause/resume of XRSession #33

Closed kyungtae closed 5 years ago

kyungtae commented 5 years ago

Because the XRSession usually use the hardware for VR/AR, sometimes the functionality for pause/resume the session is needed.

It could be useful if you want to make user to do something without exiting the ARSession. For example, if you want to make user type some messages related with an anchor on the AR environment, to pause the session during typing is nature to user, and is better in terms of battery consumption.

Additionally, it could support a better way to use the hardware properly. For example, in case of the AR based on the ArCore, if you want to get some captured images from the front camera during ARSession is ongoing, there is no way to do that now, since the access to the camera would be blocked, because the ArCore is already using the back camera. If you can pause the ARSession, this scenario could be possible.

Actually, the pause/resume is one of the basic functionality of the AR platforms such as ArCore and ArKit.

And the API could be very simple like below:

[SecureContext, Exposed=Window] interface XRSession : EventTarget {
...
  Promise<void> pause();  
  Promise<void> resume();
}
blairmacintyre commented 5 years ago

This is something that should be dealt with, but I'm curious if this will just be handled by the browser?

Assuming that AR is full screen, or immersive, it's not clear if we want to pause the underlying platform when the "world" (camera or see through) is visible. So, part of this can and should be done by the underlying platform.

The big question I'd have is, can this be automated without causing issues? For example, when the user flips to another app, on iOS/Android we know we're going into the background and can suspend/resume.

For capturing images, how are you imagining that working? On iOS, in our WebXR Viewer, if I swap to the camera, I can take a picture, and swap back to the viewer and it picks back up. Doesn't this work on ARCore apps on Android?

kyungtae commented 5 years ago

For capturing images, how are you imagining that working? On iOS, in our WebXR Viewer, if I swap to the camera, I can take a picture, and swap back to the viewer and it picks back up. Doesn't this work on ARCore apps on Android?

What I meant is not using native camera app, but using the camera in the web contents with JS APIs such as 'getUserMedia' calls. As you mentioned, if you end XRsession and get the captured image and initiate XRsession again, it's possible. However, if you want to use 'getUserMedia' without ending the XRsession, the pause/resume is needed.

blairmacintyre commented 5 years ago

A better approach would be for the APIs to actually work together. Ideally, when WebXR is open, it should be possible to access camera and sensor data (with user permission, as usual for those devices) via the API. This has been discussed separately, but no progress has been made.

Similarly, there has been discussion about adding the ability to capture the composited AR view (camera + augmentation layers) to WebXR.

It's unclear to me if pause/resume would do what you want anyway:

Anyway, I totally agree that this use case is important. I'm just not sure that this is the right approach.

TrevorFSmith commented 5 years ago

It seems like forward motion on this Issue has stalled so I'm going to close it. If you have a plan for how to make progress then ping me and we can re-open it.