Unable to Launch Android Java example for realSense

[nafidoni]

I follow the instruction for this link https://github.com/IntelRealSense/librealsense/blob/master/wrappers/android/examples/java_example/readme.md Everything was fine. Also when I connect the camera it didn't switch on and the application has just bug. Please could you help me? What should I do? Thanks.

[MartyG-RealSense]

What model of RealSense camera are you using please?

Previously, the D435i model did not yet have Android support, but this is now listed as supported for D435i in the latest 2.26.0 version of the RealSense SDK that was just released today (21 August 2019).

[nafidoni]

I am using Model: D435

[MartyG-RealSense]

Thank you very much for the confirmation of your camera model. Can you provide details of any error messages that you receive, please?

[nafidoni]

Thank you for your help. I found the solution. I was using Android 7.0 while the minimum is 7.8.

I will look for a new device and execute it.

[MartyG-RealSense]

If you are referring to the information on the Releases page, it says "7, 8" (Android 7 or Android 8), not Android version 7.8.

[nafidoni]

I test the application and that is Log

08/22 11:26:09: Launching app $ adb push /home/cedriq/Bureau/Nafi_Work/RealSense/librealsense-2.19.2/wrappers/android/examples/java_example/app/build/outputs/apk/debug/app-debug.apk /data/local/tmp/com.example.realsense_java_example $ adb shell pm install -t -r "/data/local/tmp/com.example.realsense_java_example" Success APK installed in 4 s 696 ms $ adb shell am start -n "com.example.realsense_java_example/com.example.realsense_java_example.MainActivity" -a android.intent.action.MAIN -c android.intent.category.LAUNCHER Client not ready yet..Waiting for process to come online Connected to process 9640 on device tecno-tecno_camon_cx-141.43.200.22:5555 Capturing and displaying logcat messages from application. This behavior can be disabled in the "Logcat output" section of the "Debugger" settings page. W/librs UsbUtilities: getUsbDevice: failed to locate USB device, VID: 0x8086, PID: 0x0000 D/librs UsbUtilities: hasUsbPermission W/librs UsbUtilities: hasUsbPermission: null USB device W/librs UsbUtilities: getUsbDevice: failed to locate USB device, VID: 0x8086, PID: 0x0000 D/librs UsbUtilities: grantUsbPermissions W/librs UsbUtilities: grantUsbPermissions: null USB device D/ActivityThread: EYE startEyeVerifyBroadcast packagename=com.example.realsense_java_example; ClassName=com.example.realsense_java_example.MainActivity V/PhoneWindow: DecorView setVisiblity: visibility = 4, Parent = null, this = DecorView@7b23915[],statusBarBackground visible =true,statusColor: 0xff00574b-> D/WindowClient: Add to mViews: DecorView@7b23915[MainActivity], this = android.view.WindowManagerGlobal@1e0472c D/OpenGLRenderer: Dumper init 4 threads <0x745e214f80>

is running. D/OpenGLRenderer: CanvasContext() 0x747104c000 D/GraphicBuffer: register, handle(0x746e773880) (w:1920 h:1152 s:1920 f:0x1 u:0x000100) D/ViewRootImpl[MainActivity]: hardware acceleration is enabled, this = ViewRoot{a35e5f5 com.example.realsense_java_example/com.example.realsense_java_example.MainActivity,ident = 0} V/PhoneWindow: DecorView setVisiblity: visibility = 0, Parent = ViewRoot{a35e5f5 com.example.realsense_java_example/com.example.realsense_java_example.MainActivity,ident = 0}, this = DecorView@7b23915[MainActivity],statusBarBackground visible =true,statusColor: 0xff00574b-> I/lulingjie--screenshot--observer--: observer is rigistedDecorView@7b23915[MainActivity] D/Surface: Surface::allocateBuffers(this=0x7478c54e00) D/OpenGLRenderer: CanvasContext() 0x747104c000 initialize window=0x7478c54e00, title=com.example.realsense_java_example/com.example.realsense_java_example.MainActivity I/OpenGLRenderer: Initialized EGL, version 1.4 D/OpenGLRenderer: Swap behavior 1 D/OpenGLRenderer: Created EGL context (0x7471011780) D/OpenGLRenderer: ProgramCache.init: enable enhancement 1 I/OpenGLRenderer: Get disable program binary service property (0) Initializing program atlas... I/ProgramBinary/Service: ProgramBinaryService client side disable debugging. ProgramBinaryService client side disable binary content debugging. D/ProgramBinary/Service: BpProgramBinaryService.getReady D/ProgramBinary/Service: BpProgramBinaryService.getProgramBinaryData V/InputMethodManager: onWindowFocus: null softInputMode=288 first=true flags=#81810100 I/OpenGLRenderer: Program binary detail: Binary length is 249244, program map length is 124. I/OpenGLRenderer: Succeeded to mmap program binaries. File descriptor is 62, and path is /dev/ashmem. No need to use file discriptor anymore, close fd(62). D/OpenGLRenderer: Initializing program cache from 0x0, size = -1 D/MALI: eglCreateImageKHR:513: [Crop] 0 0 1920 1152 img[1920 1152] D/Surface: Surface::connect(this=0x7478c54e00,api=1) W/libEGL: [ANDROID_RECORDABLE] format: 1 D/mali_winsys: EGLint new_window_surface(egl_winsys_display*, void*, EGLSurface, EGLConfig, egl_winsys_surface**, egl_color_buffer_format*, EGLBoolean) returns 0x3000 D/OpenGLRenderer: CacheTexture 3 upload: x, y, width height = 0, 0, 264, 273 ProgramCache.generateProgram: 0 D/GraphicBuffer: register, handle(0x745bcdc540) (w:1080 h:1920 s:1088 f:0x1 u:0x000b00) D/OpenGLRenderer: ProgramCache.generateProgram: 34359738371 D/OpenGLRenderer: ProgramCache.generateProgram: 240518168576 D/GraphicBuffer: register, handle(0x745bd6e000) (w:1080 h:1920 s:1088 f:0x1 u:0x000b00) I/librs Enumerator: onReceive: android.hardware.usb.action.USB_DEVICE_ATTACHED D/librs UsbUtilities: hasUsbPermission I/librs MessagesHandler: handleMessage: realsense device attached I/librs Enumerator: notifyOnAttach D/librs DeviceWatcher: Adding device: /dev/bus/usb/001/002 I/librs: Found UVC Device vid: id- vid- 8086 pid- ad6 mi- 0 unique_id- ? path- /dev/bus/usb/001/002 susb specification- 210 D/librs DeviceWatcher: Device: /dev/bus/usb/001/002 added successfully I/librs: Found UVC Device vid: id- vid- 8086 pid- ad6 mi- 0 unique_id- ? path- /dev/bus/usb/001/002 susb specification- 210 I/librs: Found UVC Device vid: id- vid- 8086 pid- ad6 mi- 0 unique_id- ? path- /dev/bus/usb/001/002 susb specification- 210 D/ActivityThread: EYE startEyeVerifyBroadcast packagename=com.example.realsense_java_example; ClassName=com.intel.realsense.librealsense.DeviceWatcherActivity V/InputMethodManager: onWindowFocus: null softInputMode=32 first=false flags=#81810100 D/ActivityThread: EYE startEyeVerifyBroadcast packagename=com.example.realsense_java_example; ClassName=com.example.realsense_java_example.MainActivity V/PhoneWindow: DecorView setVisiblity: visibility = 0, Parent = ViewRoot{a35e5f5 com.example.realsense_java_example/com.example.realsense_java_example.MainActivity,ident = 0}, this = DecorView@7b23915[MainActivity],statusBarBackground visible =true,statusColor: 0xff00574b-> D/GraphicBuffer: register, handle(0x745bd6f980) (w:1080 h:1920 s:1088 f:0x1 u:0x000b00) I/librs: Current firmware version: 05.09.02.00 Minimal firmware version: 05.10.03.00 I/librs: Found UVC Device vid: id- vid- 8086 pid- ad6 mi- 0 unique_id- ? path- /dev/bus/usb/001/002 susb specification- 210 I/librs: Current firmware version: 05.09.02.00 Minimal firmware version: 05.10.03.00 W/System.err: java.lang.RuntimeException: UNKNOWN_ERROR W/System.err: at com.intel.realsense.librealsense.Pipeline.nWaitForFrames(Native Method) at com.intel.realsense.librealsense.Pipeline.waitForFrames(Pipeline.java:28) at com.intel.realsense.librealsense.Pipeline.waitForFrames(Pipeline.java:24) at com.example.realsense_java_example.MainActivity.stream(MainActivity.java:99) at com.example.realsense_java_example.MainActivity.access$100(MainActivity.java:20) at com.example.realsense_java_example.MainActivity$2.run(MainActivity.java:83) at java.lang.Thread.run(Thread.java:761) W/System.err: java.lang.RuntimeException: UNKNOWN_ERROR W/System.err: at com.intel.realsense.librealsense.Pipeline.nWaitForFrames(Native Method) at com.intel.realsense.librealsense.Pipeline.waitForFrames(Pipeline.java:28) at com.intel.realsense.librealsense.Pipeline.waitForFrames(Pipeline.java:24) at com.example.realsense_java_example.MainActivity.stream(MainActivity.java:99) at com.example.realsense_java_example.MainActivity.access$100(MainActivity.java:20) at com.example.realsense_java_example.MainActivity$2.run(MainActivity.java:83) at java.lang.Thread.run(Thread.java:761) V/PhoneWindow: DecorView setVisiblity: visibility = 4, Parent = ViewRoot{a35e5f5 com.example.realsense_java_example/com.example.realsense_java_example.MainActivity,ident = 0}, this = DecorView@7b23915[MainActivity],statusBarBackground visible =true,statusColor: 0xff00574b-> D/Surface: Surface::disconnect(this=0x7478c54e00,api=1) D/GraphicBuffer: unregister, handle(0x745bcdc540) (w:1080 h:1920 s:1088 f:0x1 u:0x000b00) D/GraphicBuffer: unregister, handle(0x745bd6e000) (w:1080 h:1920 s:1088 f:0x1 u:0x000b00) D/GraphicBuffer: unregister, handle(0x745bd6f980) (w:1080 h:1920 s:1088 f:0x1 u:0x000b00) D/Surface: Surface::disconnect(this=0x7478c54e00,api=1) D/OpenGLRenderer: Destroyed EGL context (0x7471011780) D/OpenGLRenderer: Terminated EGL display (0x1) Application terminated.

[MartyG-RealSense]

Apologies for the delay in responding, I was carefully analyzing the log that you kindly provided.

As the error is related to waiting for frames, you may find some useful information in the case of a wait for frames error with Android that is in the link below.

https://github.com/IntelRealSense/librealsense/issues/2135

[nafidoni]

I read that Current firmware version: 05.09.02.00 Minimal firmware version: 05.10.03.00 Could this be the problem??

[MartyG-RealSense]

I'm not aware of a specific firmware requirement for the Android wrapper. However, having a recent firmware installed is a common fix for a lot of RealSense problems. 05.09.02.00 is one of the oldest firmwares, dating back to February 2018, not long after the camera's launch. A large number of improvements and fixes have been implemented in firmware updates since then.

[nafidoni]

Hi @MartyG-RealSense ,

Thanks you for your reply. How should I update my Firmare.

[MartyG-RealSense]

Intel recently introduced a new cross-platform firmware update system to make firmware updates easier. This includes a tool called rs-fw-update

https://dev.intelrealsense.com/docs/firmware-update-tool

If you cannot get Librealsense to work yet on your Android device, the easiest way to update the firmware may be to use a computer with Windows or Linux. This offers you a couple of ways to update the firmware:

1. Use the rs-fw-update tool mentioned above. Or:

2. Update the camera using a firmware update menu in the RealSense Viewer tool. It can be found under the 'More' option at the top of the options side-panel. You can choose a specific firmware file to update to, or choose to update to a Recommended version and let it handle the update process for you (this is the easier of the two methods).

Because the firmware driver is installed inside the camera instead of on a computer, the updated firmware will be carried to whatever device you then attach the camera to.

[vvgokul55]

Hey @MartyG-RealSense, I have been trying to connect to the Intel Realsense camera with the android device and getting UNKNOWN_ERROR at two different places based on setting the config parameter.

More details are as follows. Firmware v: 05.11.06.250 Model: D410 SDK version: 2.26.0 Android v: 9 Device: Pixel 2

Scenario 1 If I have set a config with StreamType DEPTH and COLOR at 640 X 480, I would be getting the exception while starting pipeline with that config.

Scenario 2 If I start the pipeline without the config then the exception would be thrown while waiting for the frames with timeoutMilliseconds of 1000 as @nafidoni had faced.

Can you please help me to identify the ideal resolution and StreamFormat to be setup for different StreamType as well.

Thank you in advance

[MartyG-RealSense]

@vvgokul55 At the end of May 2019, @matkatz the Android wrapper developer tried the Android wrapper with a couple of Android 9 models including the Pixel 2, so it may not be your device and its Android version that is the source of the problem.

https://github.com/IntelRealSense/librealsense/issues/4091#issuecomment-497403457

@matkatz may be the best person to address this question, as the Android wrapper is one of his specialist areas.

[vvgokul55]

@MartyG-RealSense With the help of following setup I am getting an output from the camera Device: Moto G6 Android OS version: 8.0.0 Timeout for frames: from 1000 to 2000 in milliseconds. StreamType: Depth Framerate: 0, 15 or 30 Index: 0 or -1 I am getting frames from the camera.

However when I try to add or have only a stream type Color. I am getting UNKNOWN_ERROR while starting the pipeline with the config

[MartyG-RealSense]

@vvgokul55 Your mention of the frame timeout was useful. It enabled me to find a previous case in late 2018 where the Android wrapper was reported to be displaying similar symptoms to the ones you mention - mostly fine with the depth stream but frequently timing out on the color stream.

https://github.com/IntelRealSense/librealsense/issues/2738

An Intel team member on this forum made the following suggestions on that discussion:

https://github.com/IntelRealSense/librealsense/issues/2738#issuecomment-440132617

https://github.com/IntelRealSense/librealsense/issues/2738#issuecomment-447193098

[vvgokul55]

@MartyG-RealSense, Can you please provide me a solution that can be done on the wrapper side as I am not familiar with handling the kernal layer of the android device

[MartyG-RealSense]

I do not have any hands-on experience with the Android wrapper, unfortunately. I'll try pinging @ev-mp on the Intel team, as I believe that they have had dealings with Android development.

[matkatz]

There was a bug in the Java example, please check PR #4673

@nafidoni, 05.09.02.00 FW is not supported by the Android wrapper, were you able to update? did it solve the problem?

@vvgokul55, please provide logcat.

[vvgokul55]

Hey @matkatz, attaching the log Configuration Platform: Android Device: Moto G6 (8.0) LibSense Sdk version: 2.26.0 Camera model: D415 Camera Firmware version: 05.11.06.250 Streams:

• Color (640 X 480), format: RGB8, framerate: 30, index: 0
• Depth (640 X 480), format: Z16, framerate: 30, index: 0

Wait for frames: 2000 millisec

Android Log - Stream type Color not working.txt

[vvgokul55]

Had also tested with device Samsung Galaxy Note 9(8.1.0) since it has got 3.1, Type-C 1.0 reversible connector unlike Moto G6. Also attaching the log from the device. Android Log (Note 9) - Stream type Color.txt

[matkatz]

The camera that you use is D410 which doesn't have a color sensor. You can use the Infrared stream with Y8 or RGB8 (reconstructed RGB) formats, with the same resolution and FPS as the Depth stream.

[nafidoni]

Hallo @MartyG-RealSense ,

I was not able to update the micrologiciel version.

[MartyG-RealSense]

@nafidoi Micrologiciel is French for 'firmware', apparently. :)

What device are you trying to do the firmware update on, please? (PC with Windows or Linux, an Android device,, etc).

[nafidoni]

Sorry, I was trying to update the firmware of RealSense Model D435 Current firmware version: 05.09.02.00 to Minimal firmware version: 05.10.03.00

[MartyG-RealSense]

If you have access to the RealSense Viewer software, you can update the firmware to a recommended version by going to the 'More' icon at the top of the options side-panel and selecting the 'Install Recommended Firmware' option.

[nafidoni]

Ok Thanks. I am using Linux. How can I have accesse to the realSense Viewer Software.

[MartyG-RealSense]

I am not a Linux user myself (I use Windows), but it looks as though when the Librealsense SDK is built in Linux, the binary versions of the examples and tools are placed in the folder /usr/local/bin

[vvgokul55]

@nafidoni, Can you try RS Camera app, (https://play.google.com/store/apps/details?id=com.intel.realsense.camera&hl=en) I believe it has got a feature that would update your firmware to the latest or so.

[vvgokul55]

The camera that you use is D410 which doesn't have a color sensor. You can use the Infrared stream with Y8 or RGB8 (reconstructed RGB) formats, with the same resolution and FPS as the Depth stream.

Thank you, @matkatz can you please suggest list of streams supported by D410 other than depth and infrared.

[MartyG-RealSense]

If you are able to run the tool rs-enumerate-devices then it can list the modes supported by the particular RealSense camera model currently plugged in.

https://github.com/IntelRealSense/librealsense/tree/master/tools/enumerate-devices

[matkatz]

@nafidoni please seeInstalling the packages on Linux installation instructions

@vvgokul55 what @MartyG-RealSense said is a good option. If you want to get the supported streams programmatically you can check createProfilesMap

@MartyG-RealSense Thanks!

[vvgokul55]

Thanks @matkatz & @MartyG-RealSense for the help.

[RealSenseCustomerSupport]

---

@vvgokul55 Any other questions regarding this issue? Looking forward to your reply. Thanks!

[vvgokul55]

@RealSenseCustomerSupport None from my end regarding this

[RealSenseCustomerSupport]

---

@vvgokul55 Thanks for the feedback! @nafidoni Any other questions regarding this issue? Looking forward to your reply. Thanks!