search

OSUPCVLab / mobile-ar-sensor-logger

Camera and inertial measurement unit (IMU) data logger for Android and iOS
GNU General Public License v3.0
215 stars 47 forks source link

Access to the ultra-wide angle and macro lens camera data on smartphones with multiple lenses #29

Open Jia-le-wang opened 2 years ago

Jia-le-wang commented 2 years ago

Hello, Dr. Huai,

Recently, I have been doing some research on fisheye cameras, and I want to record individual fisheye videos with MARSLogger. I purchased OPPO GT2 Pro and VIVO iQOO 10 Pro (both phones are Qualcomm chips), both of which are equipped with 150° fisheye lens. An error was reported while compiling the Marslogger2.0 code (attached picture) : Caused by: java.lang.IllegalArgumentException: path may not be null or empty string. path='null'

  1. I checked Baidu and found that the specified parameter in gradle.properties file is not correct. Could you please guide me to modify and compile it?

  2. I see you have a special on making APP for tango equipment. However, both OPPO and VIVO showed that the APP was not compatible with the device. I wonder what the difference is between an Tango APP and MARSLogger 2.0. Which APP do I need to use to remember the fisheye videos of OPPO and VIVO?

  3. I found that Huawei, Honor, and Samsung all work well with MARSLogger 2.0 and have the freedom to choose between any front and rear lenses (there are usually two front and four rear lenses). But the Mi 8 and Mi 11, as well as the newly purchased OPPO and VIVO phones with Qualcomm chips, can only choose one front and rear lens, while the rest of the ultra-wide Angle and macro lenses are not available. Is it the limit of smartphones with Qualcomm chips?

Looking forward to your reply！ error

JzHuai0108 commented 2 years ago
  1. Can you please post a screenshot of this problem? Also what branch are you compiling from? I just updated the develop branch which can show the available physical and logical cameras on the settings' camera drop list.
  2. That app is for Tango devices, such as Lenovo phab2 pro and Asus zenfone AR, which have the tango framework. No wonder that the app cannot run on ordinary android phones which does not have the tango service.
  3. I believe this has to do with the camera API rather than the processor. I recall that OPPO uses physical and logical cameras and I was not successful to record logical camera image streams because these images are synthesized. (I may be wrong here.) Can you please show me the Live camera drop list from the settings menu?
Jia-le-wang commented 2 years ago

Hi Dr. Huai，

  1. Thank you for your reply，I compiled the code from here：https://github.com/OSUPCVLab/mobile-ar-sensor-logger. Unfortunately, there is a compile error，Caused by: Java. Lang. IllegalArgumentException: path may not be null or the empty string. The path = 'null'

  2. I showed you the Live Camera drop list of my Huawei Mate30 phone. It works very well. However, on Xiaomi, OPPO and VIVO phones, they don't seem to have access to all the camera lists, just a front and a rear lens available.

In fact, OPPO, VIVO, Xiaomi and others have multiple cameras, including ultra-wide Angle, macro lens, etc. Strangely, none of the other three phones have access to the camera list except Huawei.

Jiale Wang Doctor

Beihang University @. Haidian District, Beijing, China 100191 | ---- Replied Message ---- | From | @.> | | Date | 9/21/2022 15:26 | | To | @.> | | Cc | @.> , State @.***> | | Subject | Re: [OSUPCVLab/mobile-ar-sensor-logger] An error occurred while compiling MARSLogger2.0 (Issue #29) |

Can you please post a screenshot of this problem? Also what branch are you compiling from? I just updated the develop branch which can show the available physical and logical cameras on the settings' camera drop list. That app is for Tango devices, such as Lenovo phab2 pro and Asus zenfone AR, which have the tango framework. No wonder that the app cannot run on ordinary android phones which does not have the tango service. I believe this has to do with the camera API rather than the processor. I recall that OPPO uses physical and logical cameras and I was not successful to record logical camera image streams because these images are synthesized. (I may be wrong here.) Can you please show me the Live camera drop list from the settings menu?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you modified the open/close state.Message ID: @.***>

JzHuai0108 commented 2 years ago
  1. Can you compile this branch?
  2. If you can successfully install the app built from this branch, can you show me the live camera drop list e.g., for an OPPO phone, preferably the picture?
Jia-le-wang commented 2 years ago

I still get the same error after compiling with the new branch. I attach the Android Studio version. Is it caused by the incompatibility of AS version?

Jiale Wang Doctor

Beihang University @. Haidian District, Beijing, China 100191 | ---- Replied Message ---- | From | @.> | | Date | 9/21/2022 16:48 | | To | @.> | | Cc | @.> , State @.***> | | Subject | Re: [OSUPCVLab/mobile-ar-sensor-logger] An error occurred while compiling MARSLogger2.0 (Issue #29) |

Can you compile this branch? If you can successfully install the app built from this branch, can you show me the live camera drop list e.g., for an OPPO phone, preferably the picture?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you modified the open/close state.Message ID: @.***>

JzHuai0108 commented 2 years ago

Just to make sure with a few seemingly naive questions,

  1. is your network tunneling working properly?
  2. can you confirm that the branch is develop and the commit of the branch is b26a682576591980a3f8e6fe9ce8b1cbe2f86349 (checking first 6 characters is enough)?
  3. I don't see the attachment you mentioned. If it is a picture, it is not showing up on this issue thread.
Jia-le-wang commented 2 years ago

Yes, my network is normal, and I checked the branch is b26a682576591980a3f8e6fe9ce8b1cbe2f86349. However, after I compiled the code with AS2020, the compiler still reported the same error.

Maybe if it's convenient for you, can we communicate via wechat? Dr. Huai, here's my wechat ID: 18810077255

Jiale Wang Doctor

Beihang University @. Haidian District, Beijing, China 100191 | ---- Replied Message ---- | From | @.> | | Date | 9/21/2022 17:07 | | To | @.> | | Cc | @.> , State @.***> | | Subject | Re: [OSUPCVLab/mobile-ar-sensor-logger] An error occurred while compiling MARSLogger2.0 (Issue #29) |

Just to make sure with a few seemingly naive questions,

is your network tunneling working properly? can you confirm that the branch is develop and the commit of the branch is b26a682576591980a3f8e6fe9ce8b1cbe2f86349 (checking first 6 characters is enough)? I don't see the attachment you mentioned. If it is a picture, it is not showing up on this issue thread.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you modified the open/close state.Message ID: @.***>

JzHuai0108 commented 2 years ago

Well, this problem of failing to compile has been resolved in issue 24. I think there should be a good approach to keep the release settings in build.gradle and let a user without keystore and keys compile at the same time. I will work on this.

Jia-le-wang commented 2 years ago

Thank you, Dr. Huai!

Jiale Wang Doctor

Beihang University @. Haidian District, Beijing, China 100191 | ---- Replied Message ---- | From | @.> | | Date | 9/21/2022 19:10 | | To | @.> | | Cc | @.> , State @.***> | | Subject | Re: [OSUPCVLab/mobile-ar-sensor-logger] An error occurred while compiling MARSLogger2.0 (Issue #29) |

Well, this problem has been resolved in issue 24.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you modified the open/close state.Message ID: @.***>

JzHuai0108 commented 2 years ago

As for accessing the auxiliary camera, I have found two intriguing GCam apks without requiring to root the android phones.

  1. Urnyx05. I have tested that this app can access the ultra wide angle lens on a Redmi K40 without loading configs. I think the feature of this app should be compatible with OPPO/VIVO phones. Please try it out.
  2. Wichaya. I also verified this app can access the ultra wide angle lens on the Redmi K40 without loading configs. For now, I am foraging (maybe reverse engineering) about the source code for these apks in order to emulate this feature in MARS logger.
Jia-le-wang commented 2 years ago

Thank you very much. However, these apps are not too different from the camera software of the mobile phone, they cannot record the time and IMU data of the video frame. Still looking forward to MARSLogger!

Jiale Wang Doctor

Beihang University @. Haidian District, Beijing, China 100191 | ---- Replied Message ---- | From | @.> | | Date | 9/22/2022 13:39 | | To | @.> | | Cc | @.> , State @.***> | | Subject | Re: [OSUPCVLab/mobile-ar-sensor-logger] An error occurred while compiling MARSLogger2.0 (Issue #29) |

As for accessing the auxiliary camera, I have found two intriguing GCam apks without requiring to root the android phones.

Urnyx05. I have tested that this app can access the ultra wide angle lens on a Redmi K40 without loading configs. I think the feature of this app should be compatible with OPPO/VIVO phones. Please try it out. Wichaya. I also verified this app can access the ultra wide angle lens on the Redmi K40 without loading configs. For now, I am foraging (maybe reverse engineering) about the source code for these apks in order to emulate this feature in MARS logger.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you modified the open/close state.Message ID: @.***>