search

canonical / checkbox

Checkbox is a testing framework used to validate device compatibility with Ubuntu Linux. It’s the testing tool developed for the purposes of the Ubuntu Certification program.
https://checkbox.readthedocs.io
GNU General Public License v3.0
33 stars 50 forks source link

Intel MIPI Camera can't capture the picture #1537

Open seankingyang opened 1 month ago

seankingyang commented 1 month ago

Bug Description

-----------[ Webcam still image capture test for Intel_MIPI_Camera ]------------                                                                                                                                                              
ID: com.canonical.certification::camera/still_video0                                                                                                                                                                                          
Category: com.canonical.plainbox::camera                                                                                                                                                                                                      
Purpose:

This test will check that the Intel_MIPI_Camera works

Steps:

1. Click on Test to display a still image from the camera for ten seconds.

Pick an action
    => press ENTER to continue
  c => add a comment
  s => skip this job
  q => save the session and quit
[csq]: 
... 8< -------------------------------------------------------------------------
Error: Internal data stream error.
Format: NV12 (Y/UV 4:2:0)
Resolutions: 1280x720

Failed to capture image with fswebcam, using gstreamer
Error: Internal data stream error.

Intel mipi camera can't capture the 640x480 (the script's default https://github.com/canonical/checkbox/blob/main/providers/base/bin/camera_test.py#L718-L726) which is not support on Intel MIPI camer. And I try the gst cmd:

ubuntu@ubuntu:~$ gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=10 ! "video/x-raw,width=640,height=480" ! jpegenc ! filesink location="640_480.jpeg"
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.000824281
Setting pipeline to NULL ...
Freeing pipeline ...

ubuntu@ubuntu:~$ ls -al | grep jpeg
-rw-rw-r--  1 ubuntu ubuntu        0 Oct  9 13:53 640_480.jpeg

It will generate the empty size file.

Intel MIPI camera will use the v4l2-relayd.service to transfer the fixed format and width/hight (1280x720).

$ systemctl status v4l2-relayd.service 
● v4l2-relayd.service - v4l2-relay daemon service
     Loaded: loaded (/lib/systemd/system/v4l2-relayd.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2024-09-25 22:44:39 CST; 1 week 6 days ago
   Main PID: 1203 (v4l2-relayd)
      Tasks: 4 (limit: 37880)
     Memory: 1.2G
        CPU: 39.423s
     CGroup: /system.slice/v4l2-relayd.service
             └─1203 /usr/bin/v4l2-relayd -i "icamerasrc buffer-count=7" -o "appsrc name=appsrc caps=video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! videoconvert ! v4l2sink name=v4l2sink device=/dev/video0"

Try the same gst cmd with the 1280x720:

ubuntu@ubuntu:~$ gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=10 ! "video/x-raw,width=1280,height=720" ! jpegenc ! filesink location="1280_720.jpeg"                                                                                 
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Got EOS from element "pipeline0".
Execution ended after 0:00:01.013901459
Setting pipeline to NULL ...
Freeing pipeline ...
ubuntu@ubuntu:~$ ls -al | grep 1280_720.jpeg 
-rw-rw-r--  1 ubuntu ubuntu   236634 Oct  9 13:56 1280_720.jpeg

1280_720

Although it can capture the capture the picture, but it is black.

Try the longer framebuffer and replace multiplefilesink:

ubuntu@ubuntu:~$ gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=60 ! "video/x-raw,width=1280,height=720" ! jpegenc ! multifilesink location="1280_720_multifilesink_buffer60.jpeg"                                                     

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Got EOS from element "pipeline0".
Execution ended after 0:00:02.690528436
Setting pipeline to NULL ...
Freeing pipeline ...

ubuntu@ubuntu:~$ ls -al | grep 1280_720_multifilesink_buffer60.jpeg
-rw-rw-r--  1 ubuntu ubuntu   122818 Oct  9 14:01 1280_720_multifilesink_buffer60.jpeg

1280_720_multifilesink_buffer60

Try not set the Width and height:

ubuntu@ubuntu:~$ gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=60 ! jpegenc ! multifilesink location="No_set_WH_multifilesink_buffer60.jpeg"                                                                                          
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Got EOS from element "pipeline0".
Execution ended after 0:00:02.737873971
Setting pipeline to NULL ...
Freeing pipeline ...

ubuntu@ubuntu:~$ ls -al grep No_set_WH_multifilesink_buffer60.jpeg
ls: cannot access 'grep': No such file or directory
-rw-rw-r-- 1 ubuntu ubuntu 120208 Oct  9 14:03 No_set_WH_multifilesink_buffer60.jpeg

No_set_WH_multifilesink_buffer60

If we can't get the supported_formats_info, we should not set the capability (ie. format, width, height ...etc). Intel MIPI camera need longer time to initial and open the video, so the number-buffer need to set larger.

Note: Intel MIPI camera can use fswebcam capture the picture if we set the LD_PRELOAD. But still can't set the resolutiion. (same reason with above).

Not set the LD_PRELOAD

ubuntu@ubuntu:~$ fswebcam -v -d v4l2:/dev/video0 -S 60  Not_set_LD_preload.jpeg                                                                                                                                                               
--- Opening v4l2:/dev/video0...                            
>>> Using 'v4l2' source module.                            
/dev/video0 opened.                                                                                                                                                                                                                           
src_v4l2_get_capability,87: /dev/video0 information:                                                                   
src_v4l2_get_capability,88: cap.driver: "v4l2 loopback"                                                                
src_v4l2_get_capability,89: cap.card: "Intel MIPI Camera"                                                              
src_v4l2_get_capability,90: cap.bus_info: "platform:v4l2loopback-000"                                                                                                                                                                         
src_v4l2_get_capability,91: cap.capabilities=0x85200001                                                                
src_v4l2_get_capability,92: - VIDEO_CAPTURE                                                                            
src_v4l2_get_capability,101: - READWRITE                                                                               
src_v4l2_get_capability,103: - STREAMING                                                                               
No input was specified, using the first.                                                                               
src_v4l2_set_input,181: /dev/video0: Input 0 information:                                                              
src_v4l2_set_input,182: name = "loopback"                                                                              
src_v4l2_set_input,183: type = 00000002                                                                                
src_v4l2_set_input,185: - CAMERA                           
src_v4l2_set_input,186: audioset = 00000000                                                                            
src_v4l2_set_input,187: tuner = 00000000                                                                               
src_v4l2_set_input,188: status = 00000000                                                                              
src_v4l2_set_pix_format,520: Device offers the following V4L2 pixel formats:                                                                                                                                                                  
src_v4l2_set_pix_format,533: 0: [0x3231564E] 'NV12' (Y/UV 4:2:0)                                                       
Unable to find a compatible palette format.

Set the LD_PRELOAD

ubuntu@ubuntu:~$ LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libv4l/v4l2convert.so fswebcam -v -d v4l2:/dev/video0 -S 60  SET_LD_PRELOAD.jpeg
--- Opening v4l2:/dev/video0...
>>> Using 'v4l2' source module.
/dev/video0 opened.
src_v4l2_get_capability,87: /dev/video0 information:
src_v4l2_get_capability,88: cap.driver: "v4l2 loopback"
src_v4l2_get_capability,89: cap.card: "Intel MIPI Camera"
src_v4l2_get_capability,90: cap.bus_info: "platform:v4l2loopback-000"
src_v4l2_get_capability,91: cap.capabilities=0x85200001
src_v4l2_get_capability,92: - VIDEO_CAPTURE
src_v4l2_get_capability,101: - READWRITE
src_v4l2_get_capability,103: - STREAMING
No input was specified, using the first.
src_v4l2_set_input,181: /dev/video0: Input 0 information:
src_v4l2_set_input,182: name = "loopback"
src_v4l2_set_input,183: type = 00000002
src_v4l2_set_input,185: - CAMERA
src_v4l2_set_input,186: audioset = 00000000
src_v4l2_set_input,187: tuner = 00000000
src_v4l2_set_input,188: status = 00000000
src_v4l2_set_pix_format,520: Device offers the following V4L2 pixel formats:
src_v4l2_set_pix_format,533: 0: [0x33424752] 'RGB3' (RGB3)
src_v4l2_set_pix_format,533: 1: [0x33524742] 'BGR3' (BGR3)
src_v4l2_set_pix_format,533: 2: [0x32315559] 'YU12' (YU12)
src_v4l2_set_pix_format,533: 3: [0x32315659] 'YV12' (YV12)
Using palette RGB24
Adjusting resolution from 384x288 to 1280x720.
src_v4l2_set_mmap,672: mmap information:
src_v4l2_set_mmap,673: frames=2
src_v4l2_set_mmap,722: 0 length=2764800
src_v4l2_set_mmap,722: 1 length=2764800
--- Capturing frame...
Skipping 60 frames...
Capturing 1 frames...
Captured 61 frames in 2.72 seconds. (22 fps)
--- Processing captured image...
Writing JPEG image to 'SET_LD_PRELOAD.jpeg'.
ubuntu@ubuntu:~$ ls -al ~/SET_LD_PRELOAD.jpeg 
-rw-rw-r-- 1 ubuntu ubuntu 95343 Oct  9 14:26 /home/ubuntu/SET_LD_PRELOAD.jpeg

SET_LD_PRELOAD

To Reproduce

  1. checkbox-cli run com.canonical.certification::camera-manual

Environment

Relevant log output

No response

Additional context

No response

syncronize-issues-to-jira[bot] commented 1 month ago

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CHECKBOX-1610.

This message was autogenerated