search

hiukim / mind-ar-js

Web Augmented Reality. Image Tracking, Face Tracking. Tensorflow.js
MIT License
2.24k stars 417 forks source link

Wrong Camera Detection - Multiple camera phone zoomed view issue #105

Closed andreaspaps closed 9 months ago

andreaspaps commented 2 years ago

First of all congratulations for the awesome repository!

I have a multi camera phone (mate 20 pro) and when I test my app with chrome it automatically detects the wrong camera (camera view is way too zoomed). With Firefox I can select the correct camera (back camera 3) but the video is black. The only browser that allows me to choose the correct camera and play the video correctly is Opera.

Is there any fix for this?

Sov3rain commented 2 years ago

Hey, experiencing the same behaviour here, depending on the phone model, the wrong camera is selected. on iPhone models everything is fine, but on P30 and Samsung Galaxy S20, the wide angle camera is selected.

Sov3rain commented 2 years ago

@hiukim Is there a way to manually select the camera to use?

jonomatusky commented 2 years ago

This may be helpful. I haven't tried it myself but it's on my to-do list. I'll update this thread if I make any progress.

https://github.com/AR-js-org/AR.js/issues/303

mateusmb commented 2 years ago

Hello, I fixed this on PR 203 and this is also available at my forked repo https://github.com/mateusmb/mind-ar-js. It should be straightforward to build and import the fixed code from dist/ folder.

hiukim commented 2 years ago

I guess the question is "which one is the best camera"?

Sov3rain commented 2 years ago

Of course, but we can all agree that the wide angle camera is surely not.

hiukim commented 2 years ago

@Sov3rain that's true. But how do we find the correct one programmatically? we can enumerate the devices with navigator.mediaDevices.enumerateDevices(), but then from the returned list, which one should be the default pick?

Sov3rain commented 2 years ago

@hiukim I answered you on PR #203, but to be more specific about what "device" should be used : the default lens. I'm afraid that it would need some testing to find out how each problematic phone references the "default" lens.

hiukim commented 2 years ago

@Sov3rain Yes, saw your reply. I think it's okay to have a hacky solution. But how confident are we with the solution? I personally don't have these devices to try.

If I understand correctly, PR #203 explicitly choose the 2nd back camera. The solution mentioned in ar.js looks more generic, which choose the last back camera. I understand that this could be the right camera for some of the Samsung devices you tested out.

Sov3rain commented 2 years ago

I can test the PR on a Samsung galaxy S20 and an Huawei P30, both having multiple lenses if you want. But we'll never be sure that it will always work on all devices I'm afraid.

JxstWieslaw commented 1 year ago

Hie @hiukim @mateusmb @Sov3rain

I just tried out the patch which @mateusmb but it's not working on my Huawei device. Can you assist how l can implement your fix? I downloaded your patched mind-ar-three.prod.js to my project, but the issue still persists. Please assist.

https://user-images.githubusercontent.com/72025253/213134711-356968b3-9f07-4dad-ab03-834ca52a31d5.mp4

gingemonster commented 1 year ago

Out of interest the "hack" in https://github.com/hiukim/mind-ar-js/pull/203 is very very similar (but to be clear in no way a copy) to how some of the commercial AR libraries

hiukim commented 1 year ago

I'm still not very convinced it's a robust solution, and more importantly I just don't have enough devices to test the coverage.

So I decided to add an extra parameters userDeviceId and environmentDeviceId in mindarThree constructor which allow you to specify the exact camera device you want for front facing and back facing cameras.

You can now do enumerateDevices and find your desired cameras before calling mindAR. You can refer to this example: https://github.com/hiukim/mind-ar-js/blob/master/examples/image-tracking/three.html

This feature is available in lastest version v1.2.2

sunpeng222 commented 3 months ago

I'm still not very convinced it's a robust solution, and more importantly I just don't have enough devices to test the coverage.

So I decided to add an extra parameters userDeviceId and environmentDeviceId in mindarThree constructor which allow you to specify the exact camera device you want for front facing and back facing cameras.

You can now do enumerateDevices and find your desired cameras before calling mindAR. You can refer to this example: https://github.com/hiukim/mind-ar-js/blob/master/examples/image-tracking/three.html

This feature is available in lastest version v1.2.2

May I ask if Mindar-image-aframe.djs provides such parameter input? I only saw your MindarThree related examples