search

Martin20150405 / SLAM_AR_Android

SLAM AR based on ORB_SLAM2 for Android platform (Licenced under GPL v3). Check video demo here https://www.bilibili.com/video/av56266271
126 stars 53 forks source link

Does it work with Android studio on windows? #5

Closed akash901 closed 3 years ago

akash901 commented 3 years ago

Hello @Martin20150405 , Thanks for such impressive work for developing ORB SLAM 2 port for android. I have been able to successfully test the code with Linux ubuntu 20.04 environment android studio with NDK r21e but when I am trying to test it with Android studio on Windows 10 I am facing problems.

  1. I am unable to read CameraSettings.yaml file using cv::FileStorage class. This class is always returning error while opening file whereas file exsists on my phone's storage.
  2. If I just hardcode camera parameters and try to run the code then ArUiCamActivity spawns up twice as per the logs and app kills itself after some time.

I can attach logs as well but wanted to know first if code would work with Windows 10 Android studio NDK r21e or not.

Thank you! Waiting for your response.

Martin20150405 commented 3 years ago

Hello @Martin20150405 , Thanks for such impressive work for developing ORB SLAM 2 port for android. I have been able to successfully test the code with Linux ubuntu 20.04 environment android studio with NDK r21e but when I am trying to test it with Android studio on Windows 10 I am facing problems.

  1. I am unable to read CameraSettings.yaml file using cv::FileStorage class. This class is always returning error while opening file whereas file exsists on my phone's storage.
  2. If I just hardcode camera parameters and try to run the code then ArUiCamActivity spawns up twice as per the logs and app kills itself after some time.

I can attach logs as well but wanted to know first if code would work with Windows 10 Android studio NDK r21e or not.

Thank you! Waiting for your response.

Yes. Check your line breaks, it should be LF not CRLF

akash901 commented 3 years ago

Thank you @Martin20150405 for such quick response. I changed line break from CRLF to LF using notepad++ on windows still not able to read yaml file. Please find logs attached.

$ adb shell am start -n "com.martin.ads.slamar2/com.martin.ads.ui.ModelActivity" -a android.intent.action.MAIN -c android.intent.category.LAUNCHER Connected to process 5068 on device 'oneplus-oneplus_a5010-63f09c6e'. Capturing and displaying logcat messages from application. This behavior can be disabled in the "Logcat output" section of the "Debugger" settings page. I/tin.ads.slamar: Late-enabling -Xcheck:jni E/tin.ads.slamar: Unknown bits set in runtime_flags: 0x28000 I/Perf: Connecting to perf service. I/tin.ads.slamar: [GL_OOM] ClampGrowthLimit 268435456 V/Font: Change font:1 V/Font: Default family:android.graphics.Typeface@f207bfc V/ViewRootImpl: The specified message queue synchronization barrier token has not been posted or has already been removed D/DecorView: onWindowFocusChangedFromViewRoot hasFocus: true, DecorView@1da3020[ModelActivity] D/ViewRootImpl[ModelActivity]: windowFocusChanged hasFocus=true inTouchMode=true I/AdrenoGLES: QUALCOMM build : 6209c5d, Ifdcc64d6e8 Build Date : 12/12/19 OpenGL ES Shader Compiler Version: EV031.27.05.03 Local Branch : mybranchea1c9b4a-9e05-f149-a3b2-1458d0b52ae8 Remote Branch : quic/gfx-adreno.lnx.1.0.r85-rel Remote Branch : NONE Reconstruct Branch : NOTHING Build Config : C O 9.0.3 AArch64 I/AdrenoGLES: PFP: 0x005ff112, ME: 0x005ff066 W/Gralloc3: mapper 3.x is not supported I/OpenGLRenderer: Davey! duration=894ms; Flags=1, IntendedVsync=70855427614271, Vsync=70855427614271, OldestInputEvent=9223372036854775807, NewestInputEvent=0, HandleInputStart=70855427667344, AnimationStart=70855427724010, PerformTraversalsStart=70855427728177, DrawStart=70856108175052, SyncQueued=70856162044739, SyncStart=70856162336823, IssueDrawCommandsStart=70856162385364, SwapBuffers=70856321194739, FrameCompleted=70856322529479, DequeueBufferDuration=115000, QueueBufferDuration=717000, W/ActivityThread: handleWindowVisibility: no activity for token android.os.BinderProxy@13fae7f D/ViewRootImpl[ModelActivity]: windowFocusChanged hasFocus=false inTouchMode=true W/tin.ads.slamar: Verification of boolean androidx.core.app.ComponentActivity.dispatchKeyEvent(android.view.KeyEvent) took 172.044ms W/tin.ads.slamar: Verification of com.martin.ads.ui.CameraGLView com.martin.ads.ui.ArCamUIActivity.access$000(com.martin.ads.ui.ArCamUIActivity) took 1.231s W/tin.ads.slamar: Accessing hidden method Landroid/view/View;->computeFitSystemWindows(Landroid/graphics/Rect;Landroid/graphics/Rect;)Z (greylist, reflection, allowed) Accessing hidden method Landroid/view/ViewGroup;->makeOptionalFitsSystemWindows()V (greylist, reflection, allowed) D/CameraGLViewBase: Attr count: 3 I/AppCompatViewInflater: app:theme is now deprecated. Please move to using android:theme instead. D/SlamCamActivity: OpenCV library found inside package. Using it! D/CameraGLViewBase: call checkCurrentState D/SlamCamActivity: onResume: /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/ E/JNI_PART: Model Path is /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/ Failed to open settings file at : /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/CameraSettings.yaml Camera parameters : 0.000000 0.000000 0.000000 0.000000 D/JNI_PART: /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/ORBvoc.txt /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/CameraSettings.yaml Loading ORB Vocabulary. This could take a while... Connected to process 5069 on device 'oneplus-oneplus_a5010-63f09c6e'. Capturing and displaying logcat messages from application. This behavior can be disabled in the "Logcat output" section of the "Debugger" settings page. I/Perf: Connecting to perf service. I/tin.ads.slamar: [GL_OOM] ClampGrowthLimit 268435456 V/Font: Change font:1 V/Font: Default family:android.graphics.Typeface@f207bfc W/tin.ads.slamar: Accessing hidden method Landroid/view/View;->computeFitSystemWindows(Landroid/graphics/Rect;Landroid/graphics/Rect;)Z (greylist, reflection, allowed) Accessing hidden method Landroid/view/ViewGroup;->makeOptionalFitsSystemWindows()V (greylist, reflection, allowed) D/CameraGLViewBase: Attr count: 3 I/AppCompatViewInflater: app:theme is now deprecated. Please move to using android:theme instead. D/SlamCamActivity: OpenCV library found inside package. Using it! D/CameraGLViewBase: call checkCurrentState D/SlamCamActivity: onResume: /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/ E/JNI_PART: Model Path is /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/ Failed to open settings file at : /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/CameraSettings.yaml Camera parameters : 0.000000 0.000000 0.000000 0.000000 D/JNI_PART: /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/ORBvoc.txt /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/CameraSettings.yaml Loading ORB Vocabulary. This could take a while... D/JNI_PART: Vocabulary loaded! A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 5069 (tin.ads.slamar2), pid 5069 (tin.ads.slamar2)

Martin20150405 commented 3 years ago

Thank you @Martin20150405 for such quick response. I changed line break from CRLF to LF using notepad++ on windows still not able to read yaml file. Please find logs attached.

$ adb shell am start -n "com.martin.ads.slamar2/com.martin.ads.ui.ModelActivity" -a android.intent.action.MAIN -c android.intent.category.LAUNCHER Connected to process 5068 on device 'oneplus-oneplus_a5010-63f09c6e'. Capturing and displaying logcat messages from application. This behavior can be disabled in the "Logcat output" section of the "Debugger" settings page. I/tin.ads.slamar: Late-enabling -Xcheck:jni E/tin.ads.slamar: Unknown bits set in runtime_flags: 0x28000 I/Perf: Connecting to perf service. I/tin.ads.slamar: [GL_OOM] ClampGrowthLimit 268435456 V/Font: Change font:1 V/Font: Default family:android.graphics.Typeface@f207bfc V/ViewRootImpl: The specified message queue synchronization barrier token has not been posted or has already been removed D/DecorView: onWindowFocusChangedFromViewRoot hasFocus: true, DecorView@1da3020[ModelActivity] D/ViewRootImpl[ModelActivity]: windowFocusChanged hasFocus=true inTouchMode=true I/AdrenoGLES: QUALCOMM build : 6209c5d, Ifdcc64d6e8 Build Date : 12/12/19 OpenGL ES Shader Compiler Version: EV031.27.05.03 Local Branch : mybranchea1c9b4a-9e05-f149-a3b2-1458d0b52ae8 Remote Branch : quic/gfx-adreno.lnx.1.0.r85-rel Remote Branch : NONE Reconstruct Branch : NOTHING Build Config : C O 9.0.3 AArch64 I/AdrenoGLES: PFP: 0x005ff112, ME: 0x005ff066 W/Gralloc3: mapper 3.x is not supported I/OpenGLRenderer: Davey! duration=894ms; Flags=1, IntendedVsync=70855427614271, Vsync=70855427614271, OldestInputEvent=9223372036854775807, NewestInputEvent=0, HandleInputStart=70855427667344, AnimationStart=70855427724010, PerformTraversalsStart=70855427728177, DrawStart=70856108175052, SyncQueued=70856162044739, SyncStart=70856162336823, IssueDrawCommandsStart=70856162385364, SwapBuffers=70856321194739, FrameCompleted=70856322529479, DequeueBufferDuration=115000, QueueBufferDuration=717000, W/ActivityThread: handleWindowVisibility: no activity for token android.os.BinderProxy@13fae7f D/ViewRootImpl[ModelActivity]: windowFocusChanged hasFocus=false inTouchMode=true W/tin.ads.slamar: Verification of boolean androidx.core.app.ComponentActivity.dispatchKeyEvent(android.view.KeyEvent) took 172.044ms W/tin.ads.slamar: Verification of com.martin.ads.ui.CameraGLView com.martin.ads.ui.ArCamUIActivity.access$000(com.martin.ads.ui.ArCamUIActivity) took 1.231s W/tin.ads.slamar: Accessing hidden method Landroid/view/View;->computeFitSystemWindows(Landroid/graphics/Rect;Landroid/graphics/Rect;)Z (greylist, reflection, allowed) Accessing hidden method Landroid/view/ViewGroup;->makeOptionalFitsSystemWindows()V (greylist, reflection, allowed) D/CameraGLViewBase: Attr count: 3 I/AppCompatViewInflater: app:theme is now deprecated. Please move to using android:theme instead. D/SlamCamActivity: OpenCV library found inside package. Using it! D/CameraGLViewBase: call checkCurrentState D/SlamCamActivity: onResume: /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/ E/JNI_PART: Model Path is /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/ Failed to open settings file at : /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/CameraSettings.yaml Camera parameters : 0.000000 0.000000 0.000000 0.000000 D/JNI_PART: /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/ORBvoc.txt /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/CameraSettings.yaml Loading ORB Vocabulary. This could take a while... Connected to process 5069 on device 'oneplus-oneplus_a5010-63f09c6e'. Capturing and displaying logcat messages from application. This behavior can be disabled in the "Logcat output" section of the "Debugger" settings page. I/Perf: Connecting to perf service. I/tin.ads.slamar: [GL_OOM] ClampGrowthLimit 268435456 V/Font: Change font:1 V/Font: Default family:android.graphics.Typeface@f207bfc W/tin.ads.slamar: Accessing hidden method Landroid/view/View;->computeFitSystemWindows(Landroid/graphics/Rect;Landroid/graphics/Rect;)Z (greylist, reflection, allowed) Accessing hidden method Landroid/view/ViewGroup;->makeOptionalFitsSystemWindows()V (greylist, reflection, allowed) D/CameraGLViewBase: Attr count: 3 I/AppCompatViewInflater: app:theme is now deprecated. Please move to using android:theme instead. D/SlamCamActivity: OpenCV library found inside package. Using it! D/CameraGLViewBase: call checkCurrentState D/SlamCamActivity: onResume: /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/ E/JNI_PART: Model Path is /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/ Failed to open settings file at : /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/CameraSettings.yaml Camera parameters : 0.000000 0.000000 0.000000 0.000000 D/JNI_PART: /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/ORBvoc.txt /storage/emulated/0/Android/data/com.martin.ads.slamar2/files/SLAM/CameraSettings.yaml Loading ORB Vocabulary. This could take a while... D/JNI_PART: Vocabulary loaded! A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 5069 (tin.ads.slamar2), pid 5069 (tin.ads.slamar2)

@akash901 https://github.com/Martin20150405/SLAM_AR_Android/blob/master/app/src/main/assets/config.txt

Change this file

Martin20150405 commented 3 years ago

Note you should uninstall then install this apk

akash901 commented 3 years ago

Awesome. Thanks alot @Martin20150405 . That worked like a charm. On a seperate note, I am working on integrating ORB SLAM2 for mapping and localizing in real world on one of our AR glasses. I wanted to know do you have any leads on integrating IMU data from phone with camera pose of current ORB SLAM implementation using extended kalman filter sensor fusion? I saw few open source codebase for the same but all working on linux especially with ROS and not on android. I wanted to know if you know of any good starting point for this visual inertial SLAM. I am working on this case since current localization gets lost in featureless camera view and its hard to localize post that. Many thanks once again!

akash901 commented 3 years ago

Thank you for answering my original question @Martin20150405 . I am closing this.

dheerajtibrewal commented 3 years ago

hi @akash901 , I also want to test this repo on Ubuntu 20.04 ,and as I'm a beginner in SLAM unable to find the right procedures or steps to start with, can u please guide me how u tested this code in Ubuntu, it would be really helpful for me. Thanks