search

google-ar / arcore-android-sdk

ARCore SDK for Android Studio
https://developers.google.com/ar
Other
4.95k stars 1.21k forks source link

Impact of CameraConfig on performance? #667

Open tangobravo opened 5 years ago

tangobravo commented 5 years ago

It's defined that ARCore offers 3 different CameraConfigs, all sharing the same SurfaceTexture resolution but with varying resolutions on the CPU image stream. The default on my Pixel 2 seems to be VGA.

What is not very clear to me is the impact of this choice. I understand this will reduce memory bandwidth so will help with battery life, but are there other impacts? Does ARCore itself run differently depending on this choice, or is this only provided as a convenience for users wanting to run their own computer vision code? I assume ARCore is using it internally (otherwise there would be a "none" option...) but is this, say, a tradeoff between performance and minimum detectable size for image targets? Does ARCore's CPU impact scale with this choice or is it purely about memory bandwidth?

I can live with VGA for my vision pipeline but the difference in aspect ratio and intrinsics between texture and CPU streams will necessitate more changes to integrate correctly with our cross-platform pipeline, so I'm just after a better understanding of the tradeoff.

inio commented 5 years ago

By default we attach two consumers to the camera: a 640x480 (or 640x360 on some devices) ImageReader to supply our motion tracking engine, and a nominally 1920x1080 SurfaceTexture to provide the background image for rendering. If you request a CPU image resolution other than the smallest (default), this will mean attaching a second ImageReader (third surface) to the capture session, increasing memory bandwidth used by the ISP or hardware scaler. Mostly this just means additional power usage with the extra DRAM writes, but for certain applications or on slow devices the memory bus contention could impact performance.

tangobravo commented 5 years ago

Great, thanks for the quick reply. So ARCore always uses the default size ImageReader internally, and as long as that's the specified CameraConfig the images exposed to consumers via acquireCameraImage will be using the same underlying buffers (ie zero-copy?).

Sounds like sticking with the default is definitely the way to go then for our vision code.

tangobravo commented 5 years ago

I'd suggest it would be worth adding a line or two in the docs explaining this.