raspberrypi / firmware

This repository contains pre-compiled binaries of the current Raspberry Pi kernel and modules, userspace libraries, and bootloader/GPU firmware.
5.15k stars 1.68k forks source link

Wide stripe pixel on PiZero + v1 camera on MJPEG #1494

Open tuyenld opened 3 years ago

tuyenld commented 3 years ago

Describe the bug I tried to take a JPEG image on Pi Zero + Camera v1.3, but there are pixels wide stripe on the right side.

To reproduce

WIDTH=1280
HEIGHT=720
v4l2-ctl --set-fmt-video=width=$WIDTH,height=$HEIGHT,pixelformat=MJPG
v4l2-ctl --stream-mmap=3 --stream-count=1 --stream-to="$WIDTH"_"$HEIGHT"_v4l2-ctl.jpg

then see the image output. I tested with several resolutions, but I didn't find what is the pattern of the issue happened here. Please zoom out to see the stripe on the right.

Base resolution Modified resolution
1280x720 >> stripe 1280x722 >> stripe
1280x724 (to 726, 728, .. 748) >> ok
640x480 >> stripe 640x482 >> ok
640x484 >> ok
640x478 >> stripe
1920x1080 >> ok 1920x1082 >> ok
1920x1084 >> ok
1440x1080 >> stripe 1440x1082 >> ok
1440x1084 >> ok
1440x1078 >> stripe

Expected behaviour The Pi should produce a clear image.

Actual behaviour There is a wide stripe line on the right.

System https://pastebin.com/7jPWKPdr

Logs I tried to enable bcm2835_v4l2 log, yet I didn't find anything was anormal.

root@raspberrypi:/home/pi#                                                      
root@raspberrypi:/home/pi# echo 2 | sudo tee /sys/module/bcm2835_v4l2/parameterg
2                                                                               
root@raspberrypi:/home/pi# WIDTH=640                                            
root@raspberrypi:/home/pi# HEIGHT=480                                           
root@raspberrypi:/home/pi# v4l2-ctl --set-fmt-video=width=$WIDTH,height=$HEIGHTG
root@raspberrypi:/home/pi# v4l2-ctl --stream-mmap=3 --stream-count=1 --stream-to="$WIDTH"_"$HEIGHT"_v4l2-ctl.jpg
root@raspberrypi:/home/pi# 
root@raspberrypi:/home/pi# 
root@raspberrypi:/home/pi# cat /var/log/kern.log | grep bcm2835-v4l2
Aug 20 11:51:37 raspberrypi kernel: [   17.019614] bcm2835-v4l2: scene mode selected 0, was 0
Aug 20 11:51:37 raspberrypi kernel: [   17.027967] bcm2835-v4l2: V4L2 device registered as video0 - stills mode > 1280x720
Aug 20 11:51:37 raspberrypi kernel: [   17.037321] bcm2835-v4l2: Broadcom 2835 MMAL video capture ver 0.0.2 loaded.
Aug 20 12:14:08 raspberrypi kernel: [ 1377.651679] bcm2835-v4l2: vidioc_g_fmt_vid_cap: w 1440 h 1084 field 1 pfmt 0x47504a4d bpl 0 sz_img 1566720 colorspace 0
Aug 20 12:14:08 raspberrypi kernel: [ 1377.651936] bcm2835-v4l2: Clipping/aligning 640x480 format 47504A4D
Aug 20 12:14:08 raspberrypi kernel: [ 1377.651958] bcm2835-v4l2: Not removing padding, so bytes/line = 0
Aug 20 12:14:08 raspberrypi kernel: [ 1377.651977] bcm2835-v4l2: Now 640x480 format 47504A4D
Aug 20 12:14:08 raspberrypi kernel: [ 1377.651998] bcm2835-v4l2: vidioc_try_fmt_vid_cap: w 640 h 480 field 1 pfmt 0x47504a4d bpl 0 sz_img 307200 colorspace 00
Aug 20 12:14:08 raspberrypi kernel: [ 1377.652014] bcm2835-v4l2: vid_cap - disconnect previous tunnel
Aug 20 12:14:08 raspberrypi kernel: [ 1377.653472] bcm2835-v4l2: vid_cap - set up encode comp
Aug 20 12:14:08 raspberrypi kernel: [ 1377.654711] bcm2835-v4l2: vid_cap - cur_buf.size set to 307200
Aug 20 12:14:08 raspberrypi kernel: [ 1377.654740] bcm2835-v4l2: Set dev->capture.fmt 47504A4D, 640x480, stride 0, size 307200
Aug 20 12:14:08 raspberrypi kernel: [ 1378.180485] bcm2835-v4l2: queue_setup: dev:cefafde1
Aug 20 12:14:08 raspberrypi kernel: [ 1378.180888] bcm2835-v4l2: buffer_init: dev:cefafde1, vb ec2d7ea9
Aug 20 12:14:08 raspberrypi kernel: [ 1378.181267] bcm2835-v4l2: buffer_init: dev:cefafde1, vb 63ea24d6
Aug 20 12:14:08 raspberrypi kernel: [ 1378.181628] bcm2835-v4l2: buffer_init: dev:cefafde1, vb f501dbf8
Aug 20 12:14:08 raspberrypi kernel: [ 1378.186356] bcm2835-v4l2: buffer_prepare: dev:cefafde1, vb ec2d7ea9
Aug 20 12:14:08 raspberrypi kernel: [ 1378.186402] bcm2835-v4l2: buffer_prepare: dev:cefafde1, vb 63ea24d6
Aug 20 12:14:08 raspberrypi kernel: [ 1378.186431] bcm2835-v4l2: buffer_prepare: dev:cefafde1, vb f501dbf8
Aug 20 12:14:08 raspberrypi kernel: [ 1378.186467] bcm2835-v4l2: vidioc_g_fmt_vid_cap: w 640 h 480 field 1 pfmt 0x47504a4d bpl 0 sz_img 307200 colorspace 0x10
Aug 20 12:14:08 raspberrypi kernel: [ 1378.186501] bcm2835-v4l2: buffer_queue: dev:cefafde1 buf:ec2d7ea9, idx 0
Aug 20 12:14:08 raspberrypi kernel: [ 1378.186519] bcm2835-v4l2: buffer_queue: dev:cefafde1 buf:63ea24d6, idx 1
Aug 20 12:14:08 raspberrypi kernel: [ 1378.186533] bcm2835-v4l2: buffer_queue: dev:cefafde1 buf:f501dbf8, idx 2
Aug 20 12:14:08 raspberrypi kernel: [ 1378.186547] bcm2835-v4l2: start_streaming: dev:cefafde1
Aug 20 12:14:08 raspberrypi kernel: [ 1378.258915] bcm2835-v4l2: enabled camera (refcount 1)
Aug 20 12:14:09 raspberrypi kernel: [ 1378.582122] bcm2835-v4l2: Start time 1383819681 size 8
Aug 20 12:14:09 raspberrypi kernel: [ 1378.623175] bcm2835-v4l2: buffer_cb: status:0, buf:ec2d7ea9, length:11117, flags 44, pts 1383826689
Aug 20 12:14:09 raspberrypi kernel: [ 1378.623209] bcm2835-v4l2: Convert start time 1378580459000 and 1383819681 with offset 1383826689 to 1378587467000
Aug 20 12:14:09 raspberrypi kernel: [ 1378.623228] bcm2835-v4l2: Buffer has ts 1378587467000
Aug 20 12:14:09 raspberrypi kernel: [ 1378.623887] bcm2835-v4l2: buffer_prepare: dev:cefafde1, vb ec2d7ea9
Aug 20 12:14:09 raspberrypi kernel: [ 1378.623923] bcm2835-v4l2: buffer_queue: dev:cefafde1 buf:ec2d7ea9, idx 0
Aug 20 12:14:09 raspberrypi kernel: [ 1378.624130] bcm2835-v4l2: stop_streaming: dev:cefafde1
Aug 20 12:14:09 raspberrypi kernel: [ 1378.624150] bcm2835-v4l2: stopping capturing
Aug 20 12:14:09 raspberrypi kernel: [ 1378.624428] bcm2835-v4l2: disabling connection
Aug 20 12:14:09 raspberrypi kernel: [ 1378.633662] bcm2835-v4l2: disabling port
Aug 20 12:14:09 raspberrypi kernel: [ 1378.634003] bcm2835-v4l2: buffer_cb: status:0, buf:63ea24d6, length:0, flags 0, pts 0
Aug 20 12:14:09 raspberrypi kernel: [ 1378.634052] bcm2835-v4l2: buffer_cb: status:0, buf:f501dbf8, length:0, flags 0, pts 0
Aug 20 12:14:09 raspberrypi kernel: [ 1378.634092] bcm2835-v4l2: buffer_cb: status:0, buf:ec2d7ea9, length:0, flags 0, pts 1383826689
Aug 20 12:14:09 raspberrypi kernel: [ 1378.634357] bcm2835-v4l2: Disabling camera
Aug 20 12:14:09 raspberrypi kernel: [ 1378.649651] bcm2835-v4l2: Camera refcount now 0
Aug 20 12:14:09 raspberrypi kernel: [ 1378.650300] bcm2835-v4l2: buffer_cleanup: dev:cefafde1, vb ec2d7ea9
Aug 20 12:14:09 raspberrypi kernel: [ 1378.650338] bcm2835-v4l2: buffer_cleanup: dev:cefafde1, vb 63ea24d6
Aug 20 12:14:09 raspberrypi kernel: [ 1378.650360] bcm2835-v4l2: buffer_cleanup: dev:cefafde1, vb f501dbf8
root@raspberrypi:/home/pi# 

Original issue: peterbay/uvc-gadget#5

tuyenld commented 3 years ago

This problem even was reproduced with raspivid

raspivid -fps 15 -w 640 -h 480 -cd MJPEG -o 640_480_raspivid.mjpg -t 10000 -b 25000000

Play back with VLC

vlc --demux=avformat 640_480_raspivid.mjpg

Result: 640_480_raspivid-error

Edited: However, raspistill produces a good JPEG image (without any stripe).

raspistill -w 640 -h 480 -o 640_480_still.jpg
tuyenld commented 3 years ago

@6by9, Do you have any idea? I don't know where can I start to debug this issue. Thanks!

6by9 commented 3 years ago

It sounds like one of the sensor readout modes is producing rubbish in the raw data, which then translates into rubbish in the final image. The modes are listed in https://picamera.readthedocs.io/en/latest/fov.html#sensor-modes

raspistill always uses the full resolution mode for the stills captures, and then crops it down. You can override in either raspistill or raspivid that by using the -md <mode index> option to force a particular mode. I'd suspect it's mode 5 where the issue lies, but it'd be useful if you could confirm that before I start staring at register sets.

tuyenld commented 3 years ago

Something is wrong with the sensor modes, I guess. Here is my test result (>> is my comment). I also tested with different FPS.

### 640x480 ###

$ raspivid -md 4 -fps 15 -w 640 -h 480 -cd MJPEG -o 640_480_raspivid_md4.mjpg -t 10000
>> Stripe

$ raspivid -md 5 -fps 15 -w 640 -h 480 -cd MJPEG -o 640_480_raspivid_md5.mjpg -t 10000
>> Good

$ raspivid -md 6 -fps 15 -w 640 -h 480 -cd MJPEG -o 640_480_raspivid_md6-fps15.mjpg -t 10000
>> no stripe, but color is not good

$ raspivid -md 6 -fps 45 -w 640 -h 480 -cd MJPEG -o 640_480_raspivid_md6-fps45.mjpg -t 10000
>> no stripe, but color is not good

$ raspivid -md 6 -w 640 -h 480 -cd MJPEG -o 640_480_raspivid_md6.mjpg -t 10000
>> no stripe, but color is not good

$ raspivid -md 7 -fps 15 -w 640 -h 480 -cd MJPEG -o 640_480_raspivid_md7-fps15.mjpg -t 10000
>> no stripe, but color is not good

$ raspivid -md 7 -fps 70 -w 640 -h 480 -cd MJPEG -o 640_480_raspivid_md7-fps70.mjpg -t 10000
>> no stripe, but color is not good

$ raspivid -md 7 -w 640 -h 480 -cd MJPEG -o 640_480_raspivid_md7.mjpg -t 10000
>> no stripe, but color is not good
$ 
$ 
$ ### 1270x720 ###

$ raspivid -md 3 -w 1280 -h 720 -cd MJPEG -o 1280_720_raspivid_md3.mjpg -t 10000
Camera control callback  cmd=0x4f525245mmal: No data received from sensor. Check all connections, including the Sunny one on the camera board
$ 
$ raspivid -md 3 -fps 1 -w 1280 -h 720 -cd MJPEG -o 1280_720_raspivid_md3.mjpg -t 10000
>> Good

$ raspivid -md 4 -fps 15 -w 1280 -h 720 -cd MJPEG -o 1280_720_raspivid_md4.mjpg -t 10000
>> Stripe

$ raspivid -md 5 -fps 15 -w 1280 -h 720 -cd MJPEG -o 1280_720_raspivid_md5.mjpg -t 10000
>> Stripe
tuyenld commented 3 years ago

@6by9 Is there anything else you need me to test?

6by9 commented 3 years ago

Sorry, this isn't a priority to investigate at the moment. Based on your reports I'm not convinced that it's the register setup in the sensor, so it's going to take a fair amount of investigation to work out what's going on.