Closed rsglobal closed 2 years ago
kbingham
commented
3 years ago Could you post this question to the libcamera mailling list please?
On this repository you get an audience of just me. On the libcamera mailing list you get the other libcamera developers, who have written the android layer, as well as the developers who maintain the raspberry pi pipeline handler and IPA at Raspberry Pi.
rsglobal
commented
3 years ago While debugging, got a state in which stream started for some reason. So I'm really close to get libcamera+android+rpi fully functional.
kbingham
commented
3 years ago Great! Did you identify what the change was that enabled the streaming?
rsglobal
commented
3 years ago No I haven't fixed it yet. It just looks like restarting camera app did the thing.
uajain
commented
3 years ago Hi @rsglobal
Can you briefly explain your OS setup with AOSP/android+Rpi ? I am interested.
rsglobal
commented
3 years ago Hi @uajain ,
Can you briefly explain your OS setup with AOSP/android+Rpi ? I am interested.
Not sure I can explain all setup briefly.
Here is our project: https://github.com/GloDroid/glodroid_manifest
You can look into glodroid.xml file to see the difference from AOSP.
rsglobal
commented
3 years ago Green row fixed by removing buffer height alignment in the gralloc.
rsglobal
commented
3 years ago Color correction works pretty well.
rsglobal
commented
3 years ago I've applied this series and picture freezes again. I'm attaching the logs: libcamera_v2.log
kbingham
commented
3 years ago @naushir It seems to have crashed in RPiCameraData::ispOutputDequeue
10-07 14:39:27.028 2242 2242 F DEBUG : backtrace:
10-07 14:39:27.028 2242 2242 F DEBUG : #00 pc 0000000000097c04 /apex/com.android.vndk.v31/lib64/libc++.so (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::basic_string(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)+28) (BuildId: 0db72613bf270f11efab0a156c4fe541)
10-07 14:39:27.029 2242 2242 F DEBUG : #01 pc 00000000000bf378 /vendor/lib64/libcamera.so (libcamera::RPiCameraData::ispOutputDequeue(libcamera::FrameBuffer*)+192) (BuildId: 4282ee1d889803300eeb9931b3b3b9e6)
10-07 14:39:27.029 2242 2242 F DEBUG : #02 pc 00000000000c3680 /vendor/lib64/libcamera.so (BuildId: 4282ee1d889803300eeb9931b3b3b9e6)
10-07 14:39:27.029 2242 2242 F DEBUG : #03 pc 00000000000ae300 /vendor/lib64/libcamera.so (BuildId: 4282ee1d889803300eeb9931b3b3b9e6)
10-07 14:39:27.029 2242 2242 F DEBUG : #04 pc 00000000000af37c /vendor/lib64/libcamera.so (BuildId: 4282ee1d889803300eeb9931b3b3b9e6)
10-07 14:39:27.029 2242 2242 F DEBUG : #05 pc 00000000000123c8 /vendor/lib64/libcamera-base.so (BuildId: 5609a3280ab3276ccad765984e985601)Is there an obvious path to there which may have failed to dequeue a buffer perhaps? Or something happening due to the v4l2 buffer cache issue maybe?
rsglobal
commented
3 years ago @kbingham ,
I've appended full crash and addr2line in the end of log file.
ADD: This line:
LOG(RPI, Debug) << "Stream " << stream->name() << " buffer complete"
naushir
commented
3 years ago Is there an obvious path to there which may have failed to dequeue a buffer perhaps? Or something happening due to the v4l2 buffer cache issue maybe?
The only think I can see is if stream is null, there is an assert that checks for this. This can only happen if a buffer is dequeued by the driver that pipeline handler cannot recognise. Again, this may be something to do with the v4l2 cache being under allocated, not too sure.
From the logs, this does not seem to crash immediately, but after a few restarts, is that correct?
naushir
commented
3 years ago It would be worth running with this snippet and see if you hit the warning message:
diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
index 4f6c699a4379..815ba860b126 100644
--- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
+++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
@@ -1551,8 +1551,10 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer)
}
}
- /* The buffer must belong to one of our ISP output streams. */
- ASSERT(stream);
+ if (!stream) {
+ LOG(RPI, Fatal) << "Unrecognised stream!!";
+ return;
+ }
LOG(RPI, Debug) << "Stream " << stream->name() << " buffer complete"
<< ", buffer id " << indexAttaching new logs: It looks like this time it didn't crash for some reason. But I can't see added fatal anywhere.
naushir
commented
3 years ago 10-07 14:37:46.786 381 441 E libcamera: ERROR RPISTREAM rpi_stream.cpp:239 Failed to queue buffer for ISP Output1
10-07 14:37:46.786 381 443 D libcamera: DEBUG RPiAwb awb.cpp:522 At t 6438.89 r 0.650454 b 0.895426: 0.696849
10-07 14:37:46.786 381 441 D libcamera: DEBUG RPISTREAM rpi_stream.cpp:234 Queuing buffer 1048579 for ISP Output1
10-07 14:37:46.786 381 441 E libcamera: ERROR RPISTREAM rpi_stream.cpp:239 Failed to queue buffer for ISP Output1This error shows the same buffer cache issue again, where seem to be running out of buffer slots. There seems to be quite a bit of variability in the failure cases from what I can tell, and without knowing anything about what the Android wrapper is doing, I can't really suggest much more to try.
When I am able to, perhaps this something I can recreate on my end to debug. Do you have a set of instructions for setting your environment up so I can reproduce this?
rsglobal
commented
3 years ago When I am able to, perhaps this something I can recreate on my end to debug. Do you have a set of instructions for setting your environment up so I can reproduce this?
Building AOSP has advanced hardware requirements: Minimum: 16GB of RAM (in this case all other apps must be closed) or 32gb of RAM and 300-400GB free disk space.
In case you have all this I can prepare release/instructions for you.
naushir
commented
3 years ago Even better, would you be able to create a disk image so I can flash it directly?
With all the other things going on, I am not sure when I will be able to get to this though.
rsglobal
commented
3 years ago Even better, would you be able to create a disk image so I can flash it directly?
Yes, but you won't be able to rebuild libcamera.
kbingham
commented
3 years ago At the least, I have the build capabilities for this, so instructions are probably very useful to publish somewhere if that's possible.
But I wonder if we can try to isolate the issue and replicate it in a simpler case somehow.
@rsglobal Do I recall seeing that you said the buffers /always/ use a different FD, and so are blowing through the cache because that's how we match, and that we need to check the inodes instead?
Can we verify this by printing the fd and inodes to see them get used? If the v4l2buffercache is at fault here, is that all we need to fix?
rsglobal
commented
3 years ago At the least, I have the build capabilities for this, so instructions are probably very useful to publish somewhere if that's possible.
Sure, let me just finalize v0.7.0 GloDroid release. All instructions are already HERE
@rsglobal Do I recall seeing that you said the buffers /always/ use a different FD, and so are blowing through the cache because that's how we match, and that we need to check the inodes instead?
I just assumed that buffers may use different fd if they are received from CLIENT. But honestly I do not understand what that cache are doing, and what is the info stored there and for what purpose.
Can we verify this by printing the fd and inodes to see them get used? If the v4l2buffercache is at fault here, is that all we need to fix?
Yes, ping me later please for that. (in a few weeks).
kbingham
commented
3 years ago V4L2 uses separate structures to represent the internal buffers, and we give it a dma-buf handle ( the fd ). If those are not kept associated, so the same v4l2 buffer is not used for the same underlying buffer, there is a performance penalty.
If there really is a distinct different buffer each time, we may blow through the cache, but we should instead take the performance penalty and re-use the oldest v4l2 structure (which it doesn't do now).
However, if the fd changes, but maps to the same underlying inode (same actual dma-buf) then we should use that to map the cache entry.
To know what's happening in this use case, printing all that information before it stalls would really help. I can't replicate the issue, as both locally and in chromeos where we use the android layer, the fd's are re-used so we don't exhaust the cache. This is of course 'assuming' that the underlying issue is related to that, so again - printing out the fd's to see them would really help confirm/deny the theory.
kbingham
commented
3 years ago Also, I believe we map on the FD rather than the inode currently, as otherwise - mapping on the inode would otherwise incur an extra system call to get the inode.... but perhaps it becomes unavoidable ?
rsglobal
commented
3 years ago Hi, now you can try Android-12 on RPI4 with the libcamera. Images are available under assets list: https://github.com/GloDroid/glodroid_manifest/releases/tag/v0.7.0
To build from sources please follow this manual: https://github.com/GloDroid/glodroid_manifest#fetching-android-sources
kbingham
commented
2 years ago Is this still an issue ?
Currently I have no camera picture and this:
message from the framework.
I'm on the latest libcamera & rpi-kernel v5.10.y
I will appreciate any ideas and suggestions where to look further.