Open janzwiegers opened 1 month ago
naushir
commented
1 month ago Looking at the logs, it feels like there's possibly a permissions issue, maybe the user does not have access to /dev/media* or the gralloc allocator device nodes? I'm afraid I really can't help more as I know next to nothing about the Android system environment you are running under :(
janzwiegers
commented
1 month ago Thank you for reply!
This has been an ongoing discussion on libcamera email dev list.
I have created a fork from the main libcamera repo and will start merging my changes and tests as I go.
https://github.com/RadicalES/libcamera
As I feel this is the master repo and there are plenty of other floating versions, but merely sub repos.
Main issue is I don’t know how to compile and replace a running copy of the relevant libcamera .so’s, since system/vendor files get signed in Android OS during a build.
Change, rebuild, reflash and test takes like 5-10mins.
Some smaller code issues caused problems, but main problem was this: https://github.com/RadicalES/libcamera/blob/master/src/android/mm/generic_frame_buffer_allocator.cpp
Since I know nothing about libcamera, it was really tedious debugging to figure that out.
I could not get hw_get_module to work under AOS 14.
The hardwareModule pointer was always invalid.
GRALLOC is not really the way to allocate graphics buffers under Android as I found out.
And I moved to LibUI, which adds extra layers ontop of GRALLOC.
This was really the main issue and fixed the problem right away.
I had to update and add a few more header files to satisfy working with LibUI.
Next up was to get it to work properly, it kept calling into the stub function.
And the trick was to add a Linker Flag to libui in the meson.build.
The are other smaller problems, like contiguous buffers testing on different FDs and different planes, but these I simply hacked for now, as others did as well.
My code merge started at: https://github.com/raspberry-vanilla/android_external_libcamera
Which gave a good working copy of the OS.
Then got some code from main libcamera repo.
And more from https://gitlab.baylibre.com/baylibre/ti/android/aosp/external/libcamera
And from yours https://github.com/raspberrypi/libcamera
And a lot of debugging output.
I have added a few relevant debug traces to https://github.com/RadicalES/libcamera/tree/master/reports for documentational purposes.
Currently I’m on the last issue, https://github.com/RadicalES/libcamera/blob/master/reports/debug5_thread_issue.txt
Which I have narrowed down to PostProcessorJpeg::process, and that is where I ended yesterday.
So, the last hurdle seems to be a problem converting the buffer to a JPEG if I understand it correctly.
I must still test and verify the code from my repo, as I did not really expect to get pulled in this deep into the code and changed were done in place.
And working my way back to see what changes are really needed to fix the various problems.
Cheers
Jan
From: naushir @.> Sent: Friday, June 14, 2024 10:12 AM To: raspberrypi/libcamera @.> Cc: Jan Zwiegers @.>; Author @.> Subject: Re: [raspberrypi/libcamera] Android preview streaming and image capture does not work (Issue #144)
Looking at the logs, it feels like there's possibly a permissions issue, maybe the user does not have access to /dev/media* or the gralloc allocator device nodes? I'm afraid I really can't help more as I know next to nothing about the Android system environment you are running under :(
— Reply to this email directly, view it on GitHub https://github.com/raspberrypi/libcamera/issues/144#issuecomment-2167492122 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQHOUBD6UB5EZ7OK75ICX3ZHKQTNAVCNFSM6AAAAABJD5QRBOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRXGQ4TEMJSGI . You are receiving this because you authored the thread.Message ID: @.***>
The following goes wrong when trying to capture the image while streaming the preview:
06-10 09:04:43.097 309 2929 E libcamera: ERROR Camera camera.cpp:675 Camera in Running state trying acquire() requiring state Available 06-10 09:04:43.097 309 2929 E libcamera: ERROR HAL camera_device.cpp:390 '/base/axi/pcie@120000/rp1/i2c@88000/imx708@1a': Failed to acquire the camera 06-10 09:04:43.097 309 2929 E libcamera: ERROR HAL camera3_hal.cpp:81 Failed to open camera module '0' 06-10 09:04:43.097 309 2929 E libcamera: FATAL HAL generic_frame_buffer_allocator.cpp:106 gralloc_open() failed: -87 06-10 09:04:43.098 309 2929 D libcamera: Backtrace: 06-10 09:04:43.098 309 2929 D libcamera: /vendor/lib64/hw/camera.libcamera.so(_ZN28PlatformFrameBufferAllocator7Private8allocateEiRKN9libcamera4SizeEj+0x110) [0x75835c6c90] 06-10 09:04:43.098 309 2929 D libcamera: /vendor/lib64/hw/camera.libcamera.so(_ZN12CameraStream9getBufferEv+0x74) [0x75835bea54] 06-10 09:04:43.098 309 2929 D libcamera: /vendor/lib64/hw/camera.libcamera.so(_ZN12CameraDevice21processCaptureRequestEP23camera3_capture_request+0x9b8) [0x75835b48c8] 06-10 09:04:43.098 309 2929 D libcamera: /vendor/lib64/camera.device@3.2-impl.so(_ZN7android8hardware6camera6device4V3_214implementation19CameraDeviceSession24processOneCaptureRequestERKNS3_14CaptureRequestE+0x5b4) [0x78212a9a84] 06-10 09:04:43.098 309 2929 D libcamera: /vendor/lib64/camera.device@3.2-impl.so(_ZN7android8hardware6camera6device4V3_214implementation19CameraDeviceSession21processCaptureRequestERKNS0_8hidl_vecINS3_14CaptureRequestEEERKNS6_INS3_11BufferCacheEEENSt3__18functionIFvNS> 06-10 09:04:43.098 309 2929 D libcamera: /vendor/lib64/camera.device@3.3-impl.so(+0x0) [0x781b76e1e8] 06-10 09:04:43.098 309 2929 D libcamera: /vendor/lib64/android.hardware.camera.device@3.2.so(_ZN7android8hardware6camera6device4V3_223BnHwCameraDeviceSession27_hidl_processCaptureRequestEPNS_4 06-10 09:04:43.201 3240 3240 F DEBUG : #01 pc 00000000000169ec /vendor/lib64/libcamera-base.so (libcamera::LogMessage::~LogMessage()+572) 06-10 09:04:43.201 3240 3240 F DEBUG : #02 pc 000000000003dc90 /vendor/lib64/hw/camera.libcamera.so (PlatformFrameBufferAllocator::Private::allocate(int, libcamera::Size const&, unsigned int)+272) 06-10 09:04:43.201 3240 3240 F DEBUG : #03 pc 0000000000035a54 /vendor/lib64/hw/camera.libcamera.so (CameraStream::getBuffer()+116) 06-10 09:04:43.201 3240 3240 F DEBUG : #04 pc 000000000002b8c8 /vendor/lib64/hw/camera.libcamera.so (CameraDevice::processCaptureRequest(camera3_capture_request*)+2488) 06-10 09:04:43.419 3247 3249 E libcamera: ERROR MediaDevice media_object.cpp:370 Device node /dev/video19 can't be accessed: Permission denied 06-10 09:04:43.419 3247 3249 W libcamera: WARN DeviceEnumerator device_enumerator_udev.cpp:88 Failed to populate media device /dev/media2 (rpivid), skipping 06-10 09:04:43.420 3247 3249 W libcamera: WARN DeviceEnumerator device_enumerator_udev.cpp:174 Failed to add device for '/sys/devices/platform/axi/1000800000.codec/media2', skipping 06-10 09:04:43.565 3247 3247 I CamPrvdr@2.4-legacy: Loaded "libcamera camera HALv3 module" camera module 06-10 09:04:44.286 3247 3249 E libcamera: ERROR Buffer mapped_framebuffer.cpp:226 mmap plane: fd=58 len=632 flags=3 06-10 09:04:44.286 3247 3249 E libcamera: ERROR Buffer mapped_framebuffer.cpp:226 mmap plane: fd=59 len=632 flags=3 06-10 09:04:44.286 3247 3249 E libcamera: ERROR Buffer mapped_framebuffer.cpp:226 mmap plane: fd=60 len=632 flags=3 06-10 09:04:44.286 3247 3249 E libcamera: ERROR Buffer mapped_framebuffer.cpp:226 mmap plane: fd=61 len=632 flags=3 06-10 09:04:44.287 3247 3249 E libcamera: ERROR Buffer mapped_framebuffer.cpp:226 mmap plane: fd=62 len=632 flags=3
It seems the google camera service tries to open another instance of the camera, but libcamera/RPI can only handle the streaming and not image capture at the same time. Basically it find the camera, but it is in RUNNING state and not AVAILABLE.
I have been in contact with libcamera mailing list for the last week, but they don't seem to be able to help. The latest main branch patches were applied, this does not really resolve the issue above. A normal USB camera works fine, but for this application we need a raspberry pi dual camera setup running in android. There are plenty of other little tweaks to get both cameras working under Android, for now this is the first hurdle to cross.
I'm perfectly capable of making code changes, but it is a tall order to quickly understand how libcamera and native RPI5 drivers are put together. If someone can just help me how to approach this problem, because does seem RPI related.
Please help.