the android camera2 CameraManager.getCameraCharacteristics() is not the same as ARCore camera.getImageIntrinsics()

guagua11 commented 6 months ago

SPECIFIC ISSUE ENCOUNTERED

VERSIONS USED

Android Studio: Android Studio Giraffe | 2022.3.1 Patch 1
ARCore SDK for Android: com.google.ar:core:1.39.0
Device manufacturer, model, and O/S: pixel4
Google Play Services for AR (ARCore): versionName=1.40.232630283 On Windows, use: adb shell pm dump com.google.ar.core | findstr /i "packages: versionName" On macOS, use: adb shell pm dump com.google.ar.core | egrep -i versionName\|packages:
Output of adb shell getprop ro.build.fingerprint: Android/aosp_flame/flame:10/QQ3A.200805.001/eng.root.20230829.202716:userdebug/test-keys

STEPS TO REPRODUCE THE ISSUE

1.dowload the arcore sdk 2.load the hello_ar_java demo 3.on HelloArActivity function onDrawFrame(SampleRender render) type Camera camera = frame.getCamera(); CameraIntrinsics cameraIntrinsics = camera.getImageIntrinsics(); which is the ARCore,s cameraIntrinsics and type

CameraManager manager = (CameraManager) this.getSystemService(Context.CAMERA_SERVICE); CameraCharacteristics characteristics = null; try { characteristics = manager.getCameraCharacteristics(String.valueOf(android.hardware.Camera.CameraInfo.CAMERA_FACING_BACK)); } catch (CameraAccessException e) { throw new RuntimeException(e); } float[] intrinsic; intrinsic = characteristics.get(CameraCharacteristics.LENS_INTRINSIC_CALIBRATION); which is the android camera2,s camera intrinsic,but this two intrinsic matrix is not the same as the float[fx,fy,cx,cy,s]. so, can someone tellme what,s wrong with this ,thanks a lot!

WORKAROUNDS (IF ANY)

ADDITIONAL COMMENTS

Totorro35 commented 5 months ago

Camera use by ARCore is not always the android.hardware.Camera.CameraInfo.CAMERA_FACING_BACK, it's depend of device. You can get the cameraId use by ARCore from the session session.cameraConfig.cameraId and the corresponding characteristics with characteristics = manager.getCameraCharacteristics(session.cameraConfig.cameraId

softlion commented 5 months ago

@Totorro35 Do you know for what image size the intrinsics returned by getCameraCharacteristics are ?

Totorro35 commented 5 months ago

getCameraCharacteristics returns the Characteristic of the specify CameraId. android.hardware.Camera.CameraInfo.CAMERA_FACING_BACK relates to one physical camera or logical camera (depends on devices). You can find more info about cameraId in MultiCamera API https://developer.android.com/media/camera/camera2/multi-camera.

Totorro35 commented 5 months ago

The intrinsics returns by getCameraCharacteristics can corresond to different level of zoom, aspect ratio directly managed by the manafucturer of your devices (Samsung provide a cameraId with square shape for example)

softlion commented 4 months ago

@guagua11 did you find a workaround ? It looks like ARCore is computing those intrinsics internally. It may get them from a specific camera, but the returned values are - as you said - not the same as the ones returned from the camera api.

Can't Google open source ARCore SDK now that it is no more a strategic technology ?

15kingben commented 4 months ago

Can you comment more on what is the original problem you are trying to solve? Any content placed in the AR scene should use the ARCore camera's intrinsics

softlion commented 3 months ago

I'm wondering how ARcore computes the intrinsics matrix, as it is not the same as what Camera2 api gives for the same camera

google-ar / arcore-android-sdk