search

raspberrypi / libcamera

Other
196 stars 74 forks source link

libcamerasrc on Raspberry Pi 5 with official Rasppberry Cam v2.1 doesn't understand I420 pixel format #143

Closed RSATom closed 4 weeks ago

RSATom commented 1 month ago

I've tried to build libcamera from master branch.

pi@rpi5:~/git/libcamera $ meson devenv -C build                                                                                                                                                      
[libcamera] pi@rpi5:~/git/libcamera/build $ gst-launch-1.0 -v libcamerasrc ! video/x-raw,format=I420 ! fakesink                                                                                      
Setting pipeline to PAUSED ...                                                                                                                                                                       
[0:45:15.230807221] [3697]  INFO IPAManager ipa_manager.cpp:143 libcamera is not installed. Adding '/home/pi/git/libcamera/build/src/ipa' to the IPA search path                                     
[0:45:15.233957342] [3697]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+120-eb00c13d                                                                                                         
[0:45:15.242387213] [3719]  INFO RPI pisp.cpp:695 libpisp version v1.0.5 999da5acb4f4 07-06-2024 (23:56:36)                                                                                          
[0:45:15.256831637] [3719]  INFO IPAProxy ipa_proxy.cpp:130 libcamera is not installed. Loading IPA configuration from '/home/pi/git/libcamera/src/ipa/rpi/pisp/data'                                
[0:45:15.278463328] [3719]  INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/imx219@10 to CFE device /dev/media1 and ISP device /dev/media0 using PiSP variant BCM2712_C0
Pipeline is live and does not need PREROLL ...                                                                                                                                                       
Pipeline is PREROLLED ...                                                                                                                                                                            
Setting pipeline to PLAYING ...                                                                                                                                                                      
New clock: GstSystemClock                                                                                                                                                                            
[0:45:15.279388299] [3722]  WARN V4L2 v4l2_pixelformat.cpp:344 Unsupported V4L2 pixel format RPBP                                                                                                    
[0:45:15.280156417] [3722] ERROR Stream stream.cpp:215 Range format is ambiguous                                                                                                                     
[0:45:15.280475309] [3722] ERROR Stream stream.cpp:215 Range format is ambiguous                                                                                                                     
[0:45:15.280496735] [3722] ERROR Stream stream.cpp:215 Range format is ambiguous                                                                                                                     
[0:45:15.280507457] [3722] ERROR Stream stream.cpp:215 Range format is ambiguous                                                                                                                     
ERROR: from element /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0: Internal data stream error.                                                                                                
Additional debug info:                                                                                                                                                                               
../src/gstreamer/gstlibcamerasrc.cpp(645): gst_libcamera_src_task_enter (): /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0:                                                                    
streaming stopped, reason not-negotiated (-4)                                                                                                                                                        
Execution ended after 0:00:00.002073870                                                                                                                                                              
Setting pipeline to NULL ...                                                                                                                                                                         
Freeing pipeline ...

For reference:

pi@rpi5:~ $ v4l2-ctl -d /dev/video0 --list-formats
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'YUYV' (YUYV 4:2:2)
        [1]: 'UYVY' (UYVY 4:2:2)
        [2]: 'YVYU' (YVYU 4:2:2)
        [3]: 'VYUY' (VYUY 4:2:2)
        [4]: 'RGBP' (16-bit RGB 5-6-5)
        [5]: 'RGBR' (16-bit RGB 5-6-5 BE)
        [6]: 'RGBO' (16-bit A/XRGB 1-5-5-5)
        [7]: 'RGBQ' (16-bit A/XRGB 1-5-5-5 BE)
        [8]: 'RGB3' (24-bit RGB 8-8-8)
        [9]: 'BGR3' (24-bit BGR 8-8-8)
        [10]: 'RGB4' (32-bit A/XRGB 8-8-8-8)
        [11]: 'BA81' (8-bit Bayer BGBG/GRGR)
        [12]: 'GBRG' (8-bit Bayer GBGB/RGRG)
        [13]: 'GRBG' (8-bit Bayer GRGR/BGBG)
        [14]: 'RGGB' (8-bit Bayer RGRG/GBGB)
        [15]: 'pBAA' (10-bit Bayer BGBG/GRGR Packed)
        [16]: 'pGAA' (10-bit Bayer GBGB/RGRG Packed)
        [17]: 'pgAA' (10-bit Bayer GRGR/BGBG Packed)
        [18]: 'pRAA' (10-bit Bayer RGRG/GBGB Packed)
        [19]: 'pBCC' (12-bit Bayer BGBG/GRGR Packed)
        [20]: 'pGCC' (12-bit Bayer GBGB/RGRG Packed)
        [21]: 'pgCC' (12-bit Bayer GRGR/BGBG Packed)
        [22]: 'pRCC' (12-bit Bayer RGRG/GBGB Packed)
        [23]: 'pBEE' (14-bit Bayer BGBG/GRGR Packed)
        [24]: 'pGEE' (14-bit Bayer GBGB/RGRG Packed)
        [25]: 'pgEE' (14-bit Bayer GRGR/BGBG Packed)
        [26]: 'pREE' (14-bit Bayer RGRG/GBGB Packed)
        [27]: 'BYR2' (16-bit Bayer BGBG/GRGR)
        [28]: 'GB16' (16-bit Bayer GBGB/RGRG)
        [29]: 'GR16' (16-bit Bayer GRGR/BGBG)
        [30]: 'RG16' (16-bit Bayer RGRG/GBGB)
        [31]: 'PC1R' (PiSP Bayer Compressed Format, compressed)
        [32]: 'PC1B' (PiSP Bayer Compressed Format, compressed)
        [33]: 'PC1g' (PiSP Bayer Compressed Format, compressed)
        [34]: 'PC1G' (PiSP Bayer Compressed Format, compressed)
        [35]: 'GREY' (8-bit Greyscale)
        [36]: 'Y10P' (10-bit Greyscale (MIPI Packed))
        [37]: 'Y12P' (12-bit Greyscale (MIPI Packed))
        [38]: 'Y14P' (14-bit Greyscale (MIPI Packed))
        [39]: 'Y16 ' (16-bit Greyscale)
        [40]: 'PC1M' (PiSP Bayer Compressed Format, compressed)
felixbjorkeson commented 1 month ago

This patch should probably fix it https://lists.libcamera.org/pipermail/libcamera-devel/2024-April/041661.html

RSATom commented 1 month ago

@felixbjorkeson thank you for hint! I'll try it soon.

RSATom commented 1 month ago

hm... it looks like I'll have to use libcamera v0.3 for it. But v0.3 miss some patches from current repo. Can please anybody clarify why this repo didn't rebased yet? Is there any issue with it?

naushir commented 1 month ago

We will rebase libcamera v0.3 soon, and our next libcamera release will include the fix pushed upstream.

RSATom commented 1 month ago

Then I'm going to do rebase myself and will provide info if it fixes issue or not. Thanks!

naushir commented 1 month ago

You don't need to do a full rebase. The commit in question is: https://git.libcamera.org/libcamera/libcamera.git/commit/?id=5e85157bcf156137259c57a2ebc5d5616dbb2e37

which you can cherry-pick onto this tree.

RSATom commented 1 month ago

@naushir you was right. I've tried to do rebase, and resolved some minor conflicts I've got, but on run I got 

[libcamera] pi@rpi5:~/git/libcamera/build $ gst-launch-1.0 -v libcamerasrc ! video/x-raw,format=I420 ! fakesink                                                 
Setting pipeline to PAUSED ...                                                                                                                                  
[0:16:42.186411574] [2717]  INFO IPAManager ipa_manager.cpp:143 libcamera is not installed. Adding '/home/pi/git/libcamera/build/src/ipa' to the IPA search path
[0:16:42.190401731] [2717]  INFO Camera camera_manager.cpp:313 libcamera v0.0.0+4697-07a92ae9-dirty (2024-06-12T16:55:56+07:00)                                 
[0:16:42.203172428] [2718]  INFO RPI pisp.cpp:695 libpisp version v1.0.5 999da5acb4f4 12-06-2024 (16:55:56)                                                     
[0:16:42.209182413] [2718] ERROR RPI pipeline_base.cpp:798 Failed to load a suitable IPA library                                                                
[0:16:42.209215265] [2718] ERROR RPI pisp.cpp:922 Failed to register camera imx219 6-0010: -22                                                                  
ERROR: from element /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0: Could not find any supported camera on this system.                                   
Additional debug info:                                                                                                                                          
../src/gstreamer/gstlibcamerasrc.cpp(389): gst_libcamera_src_open (): /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0:                                     
libcamera::CameraMananger::cameras() is empty                                                                                                                   
ERROR: pipeline doesn't want to preroll.                                                                                                                        
Failed to set pipeline to PAUSED.                                                                                                                               
Setting pipeline to NULL ...                                                                                                                                    
Freeing pipeline ...

maybe I resolved conflicts wrong way or it requires some additional patch....

So I've did cherry-pick of mentioned patch over current repo master branch and yes, it solved issue. Thanks!

RSATom commented 1 month ago

got some other issue when was trying to link libcamerasrc to x264enc. Pipeline just stucks by some reason:

[libcamera] pi@rpi5:~/git/libcamera/build $ gst-launch-1.0 -v libcamerasrc ! "video/x-raw, width=(int)1280, height=(int)720, framerate=(fraction)30/1, format=(string)I420, colorimetry=(string)bt709" ! queue ! x264enc ! "video/x-h264, profile=(string)constrained-baseline,level=(string)4" ! fakesink                                                                                                                                                  
Setting pipeline to PAUSED ...                                                                                                                                                                                                
[0:46:22.627604599] [4348]  INFO IPAManager ipa_manager.cpp:143 libcamera is not installed. Adding '/home/pi/git/libcamera/build/src/ipa' to the IPA search path                                                              
[0:46:22.631035024] [4348]  INFO Camera camera_manager.cpp:284 libcamera v0.0.0+4653-00f25c7a                                                                                                                                 
[0:46:22.641319079] [4349]  INFO RPI pisp.cpp:695 libpisp version v1.0.5 999da5acb4f4 12-06-2024 (17:02:43)                                                                                                                   
[0:46:22.658771409] [4349]  INFO IPAProxy ipa_proxy.cpp:130 libcamera is not installed. Loading IPA configuration from '/home/pi/git/libcamera/src/ipa/rpi/pisp/data'                                                         
[0:46:22.685572892] [4349]  INFO RPI pisp.cpp:1154 Registered camera /base/axi/pcie@120000/rp1/i2c@88000/imx219@10 to CFE device /dev/media0 and ISP device /dev/media2 using PiSP variant BCM2712_C0                         
Pipeline is live and does not need PREROLL ...                                                                                                                                                                                
Pipeline is PREROLLED ...                                                                                                                                                                                                     
Setting pipeline to PLAYING ...                                                                                                                                                                                               
New clock: GstSystemClock                                                                                                                                                                                                     
[0:46:22.686861700] [4352]  WARN V4L2 v4l2_pixelformat.cpp:344 Unsupported V4L2 pixel format RPBP                                                                                                                             
[0:46:22.688967429] [4352]  INFO Camera camera.cpp:1183 configuring streams: (0) 1280x720-YUV420                                                                                                                              
[0:46:22.689468686] [4349]  INFO RPI pisp.cpp:1450 Sensor: /base/axi/pcie@120000/rp1/i2c@88000/imx219@10 - Selected sensor format: 1920x1080-SBGGR10_1X10 - Selected CFE format: 1920x1080-PC1B                               
/GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0.GstLibcameraPad:src: caps = video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, colorimetry=(string)bt709, framerate=(fraction)30/1                      
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, colorimetry=(string)bt709, framerate=(fraction)30/1                                   
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, colorimetry=(string)bt709, framerate=(fraction)30/1                                            
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, colorimetry=(string)bt709, framerate=(fraction)30/1                                  
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, colorimetry=(string)bt709, framerate=(fraction)30/1                                             
Redistribute latency...                                                                                                                                                                                                       
/GstPipeline:pipeline0/GstX264Enc:x264enc0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, colorimetry=(string)bt709, framerate=(fraction)30/1

I'm not sure it's really related to libcamerasrc actually, but the same pipeline but with libcamerasrc replaced with videotestsrc works as expected...

also 

[libcamera] pi@rpi5:~/git/libcamera/build $ gst-launch-1.0 -v libcamerasrc  !  "video/x-raw, width=(int)1280, height=(int)720, framerate=(fraction)30/1, format=(string)NV21, colorimetry=(string)bt709" !  queue  !  videoconvert  !  x264enc ! "video/x-h264, profile=(string)constrained-baseline,level=(string)4" ! fakesink

works (it differs by requesting NV21 pixel format from libcamerasrc and using videoconvert to adopt it for x264enc input). But it's obviously bad idea to use videoconvert in that case.

naushir commented 4 weeks ago

All changes have now been integrated back to this repo. If you update your packages through apt, things should work correctly now.