mozilla-mobile / webxr-ios

An iOS app for testing WebXR
Mozilla Public License 2.0
210 stars 41 forks source link

Obtaining high-resolution image from iOS camera #110

Open lossless opened 5 years ago

lossless commented 5 years ago

Is there a way to obtain a high-res image as seen in the camera of the iOS device? I'm writing a small AR sketch using webxr-polyfill and the iOS WebXR Viewer. I'd like to obtain the current image from the camera of the iOS device, so I can use it as basis for a three.js Texture. Following the examples, I made use of XRVideoFrame and a worker object to get the current camera image's YUV data, but the image was of too low resolution. This may be more of a question than an issue, and I appreciate pointers if I'm missing an obvious answer. Thank you.

blairmacintyre commented 5 years ago

At this time, no. The method we use to get the images into Javascript (converting to base64-encoded text and injecting into Javascript) requires us to downsample. This choice was to allow a reasonable number of video frames per second.

It might be possible to change the API to support alternative sizes, which would happen at a much lower frame rate and have more overhead per frame. We aren't planning to do this, but I'll mark it as an enhancement and leave it open.