search

raspberrypi / linux

Kernel source tree for Raspberry Pi-provided kernel builds. Issues unrelated to the linux kernel should be posted on the community forum at https://forums.raspberrypi.com/
Other
11.01k stars 4.95k forks source link

Pi Camera module v2 not working in Chromium/Electron with kernel v4.4 #1498

Closed mgcrea closed 7 years ago

mgcrea commented 8 years ago

I'm struggling to make the Pi Camera module v2 work inside Chromium/Electron (while raspistill and python PiCamera do work) on an up-to-date Ubuntu Mate 16.04.

When I start electron v1.1.0 (Chrome 50.0.2661.102) on this url: https://simpl.info/getusermedia/sources I get a black square with the following logs inside /var/log/kern.log:

May 28 11:42:08 player-a8c4 kernel: [  269.703642] bcm2835_v4l2: error 0 waiting for sync completion
May 28 11:42:09 player-a8c4 kernel: [  270.708122] bcm2835-v4l2: error 0 waiting for frame completion
May 28 11:42:12 player-a8c4 kernel: [  273.702832] bcm2835_v4l2: error 0 waiting for sync completion
May 28 11:42:15 player-a8c4 kernel: [  276.705910] bcm2835_v4l2: error 0 waiting for sync completion
May 28 11:42:15 player-a8c4 kernel: [  276.705940] bcm2835-v4l2: Failed disabling camera, ret -62
May 28 11:42:15 player-a8c4 kernel: [  276.705951] bcm2835-v4l2: Failed to disable camera
May 28 11:42:15 player-a8c4 kernel: [  276.705961] ------------[ cut here ]------------
May 28 11:42:15 player-a8c4 kernel: [  276.705996] WARNING: CPU: 3 PID: 5702 at drivers/media/v4l2-core/videobuf2-core.c:1670 __vb2_queue_cancel+0xfc/0x160 [videobuf2_core]()
May 28 11:42:15 player-a8c4 kernel: [  276.706006] Modules linked in: tun rfcomm fuse xt_multiport xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter ip_tables x_tables nf_nat nf_conntrack br_netfilter bridge stp llc overlay bnep hci_uart btbcm bluetooth evdev joydev brcmfmac brcmutil cfg80211 rfkill bcm2835_gpiomem bcm2835_wdt rpi_ft5406 uio_pdrv_genirq rpi_backlight uio snd_bcm2835 snd_pcm_oss snd_mixer_oss snd_seq_dummy snd_pcm snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_seq_device snd_timer snd bcm2835_v4l2 videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_core v4l2_common videodev media ipv6
May 28 11:42:15 player-a8c4 kernel: [  276.706254] CPU: 3 PID: 5702 Comm: V4L2CaptureThre Not tainted 4.4.11-v7+ #888
May 28 11:42:15 player-a8c4 kernel: [  276.706262] Hardware name: BCM2709
May 28 11:42:15 player-a8c4 kernel: [  276.706303] [<80018724>] (unwind_backtrace) from [<80014058>] (show_stack+0x20/0x24)
May 28 11:42:15 player-a8c4 kernel: [  276.706329] [<80014058>] (show_stack) from [<803205a4>] (dump_stack+0xd4/0x118)
May 28 11:42:15 player-a8c4 kernel: [  276.706354] [<803205a4>] (dump_stack) from [<80025300>] (warn_slowpath_common+0x98/0xc8)
May 28 11:42:15 player-a8c4 kernel: [  276.706379] [<80025300>] (warn_slowpath_common) from [<800253ec>] (warn_slowpath_null+0x2c/0x34)
May 28 11:42:15 player-a8c4 kernel: [  276.706405] [<800253ec>] (warn_slowpath_null) from [<7f0b328c>] (__vb2_queue_cancel+0xfc/0x160 [videobuf2_core])
May 28 11:42:15 player-a8c4 kernel: [  276.706439] [<7f0b328c>] (__vb2_queue_cancel [videobuf2_core]) from [<7f0b3878>] (vb2_core_streamoff+0x44/0xac [videobuf2_core])
May 28 11:42:15 player-a8c4 kernel: [  276.706473] [<7f0b3878>] (vb2_core_streamoff [videobuf2_core]) from [<7f0bd5dc>] (vb2_streamoff+0x40/0x60 [videobuf2_v4l2])
May 28 11:42:15 player-a8c4 kernel: [  276.706504] [<7f0bd5dc>] (vb2_streamoff [videobuf2_v4l2]) from [<7f0bd64c>] (vb2_ioctl_streamoff+0x50/0x54 [videobuf2_v4l2])
May 28 11:42:15 player-a8c4 kernel: [  276.706553] [<7f0bd64c>] (vb2_ioctl_streamoff [videobuf2_v4l2]) from [<7f079d40>] (v4l_streamoff+0x28/0x2c [videodev])
May 28 11:42:15 player-a8c4 kernel: [  276.706609] [<7f079d40>] (v4l_streamoff [videodev]) from [<7f07de68>] (__video_do_ioctl+0x2a0/0x314 [videodev])
May 28 11:42:15 player-a8c4 kernel: [  276.706664] [<7f07de68>] (__video_do_ioctl [videodev]) from [<7f07d720>] (video_usercopy+0x188/0x60c [videodev])
May 28 11:42:15 player-a8c4 kernel: [  276.706724] [<7f07d720>] (video_usercopy [videodev]) from [<7f07dbc4>] (video_ioctl2+0x20/0x24 [videodev])
May 28 11:42:15 player-a8c4 kernel: [  276.706780] [<7f07dbc4>] (video_ioctl2 [videodev]) from [<7f078720>] (v4l2_ioctl+0xd4/0xe8 [videodev])
May 28 11:42:15 player-a8c4 kernel: [  276.706821] [<7f078720>] (v4l2_ioctl [videodev]) from [<80169288>] (do_vfs_ioctl+0x424/0x614)
May 28 11:42:15 player-a8c4 kernel: [  276.706843] [<80169288>] (do_vfs_ioctl) from [<801694bc>] (SyS_ioctl+0x44/0x6c)
May 28 11:42:15 player-a8c4 kernel: [  276.706865] [<801694bc>] (SyS_ioctl) from [<8000fb40>] (ret_fast_syscall+0x0/0x1c)
May 28 11:42:15 player-a8c4 kernel: [  276.706887] ---[ end trace 92cd1901b66857da ]---
# uname -a
Linux player-a8c4 4.4.11-v7+ #888 SMP Mon May 23 20:10:33 BST 2016 armv7l armv7l armv7l GNU/Linux

After that, the camera module seems stuck and any further raspistill won't work.

Any ideas?

Might be related to #849?

6by9 commented 8 years ago

Why duplicate the issue - #1499 ? Has this worked on the V1 camera? It sounds like a buffering issue, but can't tell from those logs. Please run echo 2 | sudo tee /sys/module/bcm2835_v4l2/parameters/debug before the run to turn on extra logging, and then post the results from the kernel log when wedged.

Unrelated to #849 - GStreamer is objecting to particular formatting of an H264 elementary stream.

817 is closed - there was a warning coming up on every stop due to not releasing buffers. You see a similar warning as the GPU has apparently wedged and hasn't returned a buffer.

mgcrea commented 8 years ago

@6by9 sorry for the duplicate (and pinging an old unrelated issue), I had connectivity issues.

Did not have the chance to test with a V1 camera.

Here is the full log with debug=2:

May 28 14:10:15 player-a8c4 kernel: [  195.634980] bcm2835-v4l2: Clipping/aligning 0x0 format 32315559
May 28 14:10:15 player-a8c4 kernel: [  195.635003] bcm2835-v4l2: Now 16x16 format 32315559
May 28 14:10:15 player-a8c4 kernel: [  195.635013] bcm2835-v4l2: vidioc_try_fmt_vid_cap: w 16 h 16 field 1 pfmt 0x32315559 bpl 16 sz_img 768 colorspace 0x1 priv 0
May 28 14:10:15 player-a8c4 kernel: [  195.635019] bcm2835-v4l2: vid_cap - disconnect previous tunnel
May 28 14:10:15 player-a8c4 kernel: [  195.643414] bcm2835-v4l2: Set dev->capture.fmt 30323449, 16x16, stride 16, size 768
May 28 14:10:15 player-a8c4 kernel: [  195.643458] bcm2835-v4l2: Set fps range to 300000/10000 to 300000/10000
May 28 14:10:15 player-a8c4 kernel: [  195.643826] bcm2835-v4l2: queue_setup: dev:b4b28800
May 28 14:10:15 player-a8c4 kernel: [  195.643947] bcm2835-v4l2: buffer_prepare: dev:b4b28800
May 28 14:10:15 player-a8c4 kernel: [  195.643982] bcm2835-v4l2: buffer_prepare: dev:b4b28800
May 28 14:10:15 player-a8c4 kernel: [  195.644007] bcm2835-v4l2: buffer_prepare: dev:b4b28800
May 28 14:10:15 player-a8c4 kernel: [  195.644032] bcm2835-v4l2: buffer_prepare: dev:b4b28800
May 28 14:10:15 player-a8c4 kernel: [  195.644047] bcm2835-v4l2: buffer_queue: dev:b4b28800 buf:b3330e00
May 28 14:10:15 player-a8c4 kernel: [  195.644054] bcm2835-v4l2: buffer_queue: dev:b4b28800 buf:b3331000
May 28 14:10:15 player-a8c4 kernel: [  195.644059] bcm2835-v4l2: buffer_queue: dev:b4b28800 buf:b3330800
May 28 14:10:15 player-a8c4 kernel: [  195.644065] bcm2835-v4l2: buffer_queue: dev:b4b28800 buf:b3331e00
May 28 14:10:15 player-a8c4 kernel: [  195.644071] bcm2835-v4l2: start_streaming: dev:b4b28800
May 28 14:10:15 player-a8c4 kernel: [  195.996552] bcm2835-v4l2: enabled camera (refcount 1)
May 28 14:10:16 player-a8c4 kernel: [  196.306655] bcm2835-v4l2: Start time 200600607 size 8
May 28 14:10:18 player-a8c4 kernel: [  198.315059] bcm2835-v4l2: stop_streaming: dev:b4b28800
May 28 14:10:18 player-a8c4 kernel: [  198.315087] bcm2835-v4l2: stopping capturing
May 28 14:10:21 player-a8c4 kernel: [  201.316991] bcm2835_v4l2: error 0 waiting for sync completion
May 28 14:10:22 player-a8c4 kernel: [  202.316609] bcm2835-v4l2: error 0 waiting for frame completion
May 28 14:10:22 player-a8c4 kernel: [  202.316632] bcm2835-v4l2: disabling connection
May 28 14:10:25 player-a8c4 kernel: [  205.316666] bcm2835_v4l2: error 0 waiting for sync completion
May 28 14:10:25 player-a8c4 kernel: [  205.316693] bcm2835-v4l2: Disabling camera
May 28 14:10:28 player-a8c4 kernel: [  208.317281] bcm2835_v4l2: error 0 waiting for sync completion
May 28 14:10:28 player-a8c4 kernel: [  208.317323] bcm2835-v4l2: Failed disabling camera, ret -62
May 28 14:10:28 player-a8c4 kernel: [  208.317335] bcm2835-v4l2: Failed to disable camera
May 28 14:10:28 player-a8c4 kernel: [  208.317356] ------------[ cut here ]------------
May 28 14:10:28 player-a8c4 kernel: [  208.317414] WARNING: CPU: 1 PID: 4409 at drivers/media/v4l2-core/videobuf2-core.c:1670 __vb2_queue_cancel+0xfc/0x160 [videobuf2_core]()
May 28 14:10:28 player-a8c4 kernel: [  208.317426] Modules linked in: rfcomm tun fuse xt_multiport xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter ip_tables x_tables nf_nat nf_conntrack br_netfilter bridge stp llc overlay bnep hci_uart btbcm bluetooth evdev joydev brcmfmac brcmutil cfg80211 rfkill bcm2835_gpiomem rpi_ft5406 bcm2835_wdt rpi_backlight uio_pdrv_genirq uio snd_bcm2835 snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_seq_device snd_timer snd bcm2835_v4l2 videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_core v4l2_common videodev media ipv6
May 28 14:10:28 player-a8c4 kernel: [  208.317836] CPU: 1 PID: 4409 Comm: V4L2CaptureThre Not tainted 4.4.11-v7+ #888
May 28 14:10:28 player-a8c4 kernel: [  208.317850] Hardware name: BCM2709
May 28 14:10:28 player-a8c4 kernel: [  208.317903] [<80018724>] (unwind_backtrace) from [<80014058>] (show_stack+0x20/0x24)
May 28 14:10:28 player-a8c4 kernel: [  208.317938] [<80014058>] (show_stack) from [<803205a4>] (dump_stack+0xd4/0x118)
May 28 14:10:28 player-a8c4 kernel: [  208.317986] [<803205a4>] (dump_stack) from [<80025300>] (warn_slowpath_common+0x98/0xc8)
May 28 14:10:28 player-a8c4 kernel: [  208.318026] [<80025300>] (warn_slowpath_common) from [<800253ec>] (warn_slowpath_null+0x2c/0x34)
May 28 14:10:28 player-a8c4 kernel: [  208.318084] [<800253ec>] (warn_slowpath_null) from [<7f0b328c>] (__vb2_queue_cancel+0xfc/0x160 [videobuf2_core])
May 28 14:10:28 player-a8c4 kernel: [  208.318126] [<7f0b328c>] (__vb2_queue_cancel [videobuf2_core]) from [<7f0b3878>] (vb2_core_streamoff+0x44/0xac [videobuf2_core])
May 28 14:10:28 player-a8c4 kernel: [  208.318160] [<7f0b3878>] (vb2_core_streamoff [videobuf2_core]) from [<7f0bd5dc>] (vb2_streamoff+0x40/0x60 [videobuf2_v4l2])
May 28 14:10:28 player-a8c4 kernel: [  208.318216] [<7f0bd5dc>] (vb2_streamoff [videobuf2_v4l2]) from [<7f0bd64c>] (vb2_ioctl_streamoff+0x50/0x54 [videobuf2_v4l2])
May 28 14:10:28 player-a8c4 kernel: [  208.318300] [<7f0bd64c>] (vb2_ioctl_streamoff [videobuf2_v4l2]) from [<7f079d40>] (v4l_streamoff+0x28/0x2c [videodev])
May 28 14:10:28 player-a8c4 kernel: [  208.318381] [<7f079d40>] (v4l_streamoff [videodev]) from [<7f07de68>] (__video_do_ioctl+0x2a0/0x314 [videodev])
May 28 14:10:28 player-a8c4 kernel: [  208.318471] [<7f07de68>] (__video_do_ioctl [videodev]) from [<7f07d720>] (video_usercopy+0x188/0x60c [videodev])
May 28 14:10:28 player-a8c4 kernel: [  208.318556] [<7f07d720>] (video_usercopy [videodev]) from [<7f07dbc4>] (video_ioctl2+0x20/0x24 [videodev])
May 28 14:10:28 player-a8c4 kernel: [  208.318624] [<7f07dbc4>] (video_ioctl2 [videodev]) from [<7f078720>] (v4l2_ioctl+0xd4/0xe8 [videodev])
May 28 14:10:28 player-a8c4 kernel: [  208.318682] [<7f078720>] (v4l2_ioctl [videodev]) from [<80169288>] (do_vfs_ioctl+0x424/0x614)
May 28 14:10:28 player-a8c4 kernel: [  208.318727] [<80169288>] (do_vfs_ioctl) from [<801694bc>] (SyS_ioctl+0x44/0x6c)
May 28 14:10:28 player-a8c4 kernel: [  208.318767] [<801694bc>] (SyS_ioctl) from [<8000fb40>] (ret_fast_syscall+0x0/0x1c)
May 28 14:10:28 player-a8c4 kernel: [  208.318785] ---[ end trace 078ff5d67f121bab ]---

Also tried with latest electron v1.2.0 (Chrome 51.0.2704.54), same error.

# v4l2-ctl  --all
Driver Info (not using libv4l2):
    Driver name   : bm2835 mmal
    Card type     : mmal service -1820074580.-18200
    Bus info      : platform:bcm2835-v4l2
    Driver version: 4.4.11
    Capabilities  : 0x85200005
        Video Capture
        Video Overlay
        Read/Write
        Streaming
        Extended Pix Format
        Device Capabilities
    Device Caps   : 0x05200005
        Video Capture
        Video Overlay
        Read/Write
        Streaming
        Extended Pix Format
Priority: 2
Video input : 0 (Camera 0: ok)
Format Video Capture:
    Width/Height      : 16/16
    Pixel Format      : 'YU12'
    Field             : None
    Bytes per Line    : 16
    Size Image        : 768
    Colorspace        : SMPTE 170M
    Transfer Function : Default
    YCbCr Encoding    : Default
    Quantization      : Default
    Flags             : 
Format Video Overlay:
    Left/Top    : 150/50
    Width/Height: 1024/768
    Field       : None
    Chroma Key  : 0x00000000
    Global Alpha: 0xff
    Clip Count  : 0
    Clip Bitmap : No
Framebuffer Format:
    Capability    : Extern Overlay
            Global Alpha
    Flags         : Overlay Matches Capture/Output Size
    Width         : 16
    Height        : 16
    Pixel Format  : 'YU12'
Streaming Parameters Video Capture:
    Capabilities     : timeperframe
    Frames per second: 30.000 (300000/10000)
    Read buffers     : 1

User Controls

                     brightness (int)    : min=0 max=100 step=1 default=50 value=50 flags=slider
                       contrast (int)    : min=-100 max=100 step=1 default=0 value=0 flags=slider
                     saturation (int)    : min=-100 max=100 step=1 default=0 value=0 flags=slider
                    red_balance (int)    : min=1 max=7999 step=1 default=1000 value=1000 flags=slider
                   blue_balance (int)    : min=1 max=7999 step=1 default=1000 value=1000 flags=slider
                horizontal_flip (bool)   : default=0 value=0
                  vertical_flip (bool)   : default=0 value=0
           power_line_frequency (menu)   : min=0 max=3 default=1 value=1
                      sharpness (int)    : min=-100 max=100 step=1 default=0 value=0 flags=slider
                  color_effects (menu)   : min=0 max=15 default=0 value=0
                         rotate (int)    : min=0 max=360 step=90 default=0 value=0
             color_effects_cbcr (int)    : min=0 max=65535 step=1 default=32896 value=32896

Codec Controls

             video_bitrate_mode (menu)   : min=0 max=1 default=0 value=0 flags=update
                  video_bitrate (int)    : min=25000 max=25000000 step=25000 default=10000000 value=10000000
         repeat_sequence_header (bool)   : default=0 value=0
            h264_i_frame_period (int)    : min=0 max=2147483647 step=1 default=60 value=60
                     h264_level (menu)   : min=0 max=11 default=11 value=11
                   h264_profile (menu)   : min=0 max=4 default=4 value=4

Camera Controls

                  auto_exposure (menu)   : min=0 max=3 default=0 value=0
         exposure_time_absolute (int)    : min=1 max=10000 step=1 default=1000 value=1000
     exposure_dynamic_framerate (bool)   : default=0 value=0
             auto_exposure_bias (intmenu): min=0 max=24 default=12 value=12
      white_balance_auto_preset (menu)   : min=0 max=9 default=1 value=1
            image_stabilization (bool)   : default=0 value=0
                iso_sensitivity (intmenu): min=0 max=4 default=0 value=0
           iso_sensitivity_auto (menu)   : min=0 max=1 default=1 value=1
         exposure_metering_mode (menu)   : min=0 max=2 default=0 value=0
                     scene_mode (menu)   : min=0 max=13 default=0 value=0

JPEG Compression Controls

            compression_quality (int)    : min=1 max=100 step=1 default=30 value=30
6by9 commented 8 years ago

Your app is asking for a 0x0 YU12 image - that sounds a fairly unhelpful resolution, even if it is relying on V4L2 to clip it to a valid range (16x16 is the minimum supposedly supported on Pi)

Admittedly something odd is going on on the GPU side - 16x16 doesn't return any buffers. 24x16 and 32x16 do. 18x16 doesn't. I haven't run any extra tests yet to find exactly the limit. I can hazard a guess at what is going on, but would need to check a few things to know for sure. I suspect it would fail on a V1 sensor too as my hunch is that it is a framework thing.

I would however suggest you investigate why your application is asking for such a low resolution. That sounds like a bug on that side, possibly down to not supporting drivers that advertise stepwise resolution support from VIDIOC_ENUM_FRAMESIZES (see http://hverkuil.home.xs4all.nl/spec/media.html#vidioc-enum-framesizes)

(edited to correct the requested resolution)

mgcrea commented 8 years ago

I don't think there is anything specific to the app itself, here is the JavaScript source code, and it is basically just Chrome's navigator.getUserMedia, that does not look like it allows any format/size customization.

6by9 commented 8 years ago

From the docs section "Examples/Width and height"

if (navigator.getUserMedia) {
   navigator.getUserMedia({ audio: true, video: { width: 1280, height: 720 } },
      function(stream) {
         var video = document.querySelector('video');
         video.src = window.URL.createObjectURL(stream);
         video.onloadedmetadata = function(e) {
           video.play();
         };
      },

So you can ask for specific width and height.

Note too the main banner heading:

Deprecated This feature has been removed from the Web standards. Though some browsers may still support it, it is in the process of being dropped. Do not use it in old or new projects. Pages or Web apps using it may break at any time.

I can fix the underlying issue, but you'll get a 16x16 image streamed into your browser - it will look terrible, and that is a result of the app asking for something silly.

mgcrea commented 8 years ago

Indeed, I refered/linked to an incomplete translated page, will try the navigator.mediaDevices.getUserMedia with different sizes and report back. Thanks!

mgcrea commented 8 years ago

Still no luck tonight with this code (working thanks to this polyfill):

// Prefer camera resolution nearest to 1280x720.
var constraints = { audio: true, video: { width: 1280, height: 720 } };

navigator.mediaDevices.getUserMedia(constraints)
.then(function(stream) {
  var video = document.querySelector('video');
  video.src = window.URL.createObjectURL(stream);
  video.onloadedmetadata = function(e) {
    video.play();
  };
})
.catch(function(err) {
  console.log(err.name + ": " + err.message);
});

Got the following output:

May 30 22:57:19 player-a8c4 kernel: [   65.417346] bcm2835-v4l2: Clipping/aligning 0x0 format 32315559
May 30 22:57:19 player-a8c4 kernel: [   65.417372] bcm2835-v4l2: Now 16x16 format 32315559
May 30 22:57:19 player-a8c4 kernel: [   65.417381] bcm2835-v4l2: vidioc_try_fmt_vid_cap: w 16 h 16 field 1 pfmt 0x32315559 bpl 16 sz_img 768 colorspace 0x1 priv 0
May 30 22:57:19 player-a8c4 kernel: [   65.417387] bcm2835-v4l2: vid_cap - disconnect previous tunnel
May 30 22:57:19 player-a8c4 kernel: [   65.422739] bcm2835-v4l2: Set dev->capture.fmt 30323449, 16x16, stride 16, size 768
May 30 22:57:19 player-a8c4 kernel: [   65.422775] bcm2835-v4l2: Set fps range to 300000/10000 to 300000/10000
May 30 22:57:19 player-a8c4 kernel: [   65.422992] bcm2835-v4l2: queue_setup: dev:b50f4000
May 30 22:57:19 player-a8c4 kernel: [   65.423089] bcm2835-v4l2: buffer_prepare: dev:b50f4000
May 30 22:57:19 player-a8c4 kernel: [   65.423119] bcm2835-v4l2: buffer_prepare: dev:b50f4000
May 30 22:57:19 player-a8c4 kernel: [   65.423143] bcm2835-v4l2: buffer_prepare: dev:b50f4000
May 30 22:57:19 player-a8c4 kernel: [   65.423166] bcm2835-v4l2: buffer_prepare: dev:b50f4000
May 30 22:57:19 player-a8c4 kernel: [   65.423177] bcm2835-v4l2: buffer_queue: dev:b50f4000 buf:90df6e00
May 30 22:57:19 player-a8c4 kernel: [   65.423183] bcm2835-v4l2: buffer_queue: dev:b50f4000 buf:90df7600
May 30 22:57:19 player-a8c4 kernel: [   65.423189] bcm2835-v4l2: buffer_queue: dev:b50f4000 buf:90df6000
May 30 22:57:19 player-a8c4 kernel: [   65.423194] bcm2835-v4l2: buffer_queue: dev:b50f4000 buf:90df6c00
May 30 22:57:19 player-a8c4 kernel: [   65.423199] bcm2835-v4l2: start_streaming: dev:b50f4000
May 30 22:57:19 player-a8c4 kernel: [   65.776960] bcm2835-v4l2: enabled camera (refcount 1)
May 30 22:57:20 player-a8c4 kernel: [   66.079716] bcm2835-v4l2: Start time 70358526 size 8
May 30 22:57:22 player-a8c4 kernel: [   68.087291] bcm2835-v4l2: stop_streaming: dev:b50f4000
May 30 22:57:22 player-a8c4 kernel: [   68.087305] bcm2835-v4l2: stopping capturing
May 30 22:57:25 player-a8c4 kernel: [   71.079696] bcm2835_v4l2: error 0 waiting for sync completion
May 30 22:57:26 player-a8c4 kernel: [   72.079765] bcm2835-v4l2: error 0 waiting for frame completion
May 30 22:57:26 player-a8c4 kernel: [   72.079789] bcm2835-v4l2: disabling connection
May 30 22:57:29 player-a8c4 kernel: [   75.079715] bcm2835_v4l2: error 0 waiting for sync completion
May 30 22:57:29 player-a8c4 kernel: [   75.079735] bcm2835-v4l2: Disabling camera
May 30 22:57:32 player-a8c4 kernel: [   78.079750] bcm2835_v4l2: error 0 waiting for sync completion
May 30 22:57:32 player-a8c4 kernel: [   78.079768] bcm2835-v4l2: Failed disabling camera, ret -62
May 30 22:57:32 player-a8c4 kernel: [   78.079773] bcm2835-v4l2: Failed to disable camera
May 30 22:57:32 player-a8c4 kernel: [   78.079779] ------------[ cut here ]------------
May 30 22:57:32 player-a8c4 kernel: [   78.079798] WARNING: CPU: 0 PID: 2728 at drivers/media/v4l2-core/videobuf2-core.c:1670 __vb2_queue_cancel+0xfc/0x160 [videobuf2_core]()
May 30 22:57:32 player-a8c4 kernel: [   78.079803] Modules linked in: tun rfcomm fuse xt_multiport xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 bnep iptable_nat hci_uart nf_conntrack_ipv4 btbcm bluetooth nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter ip_tables x_tables nf_nat nf_conntrack br_netfilter bridge stp llc overlay evdev joydev brcmfmac brcmutil cfg80211 rfkill bcm2835_wdt bcm2835_gpiomem rpi_ft5406 rpi_backlight uio_pdrv_genirq uio snd_bcm2835 snd_pcm_oss snd_mixer_oss snd_pcm snd_seq_dummy snd_seq_oss snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_seq_device snd_timer snd bcm2835_v4l2 videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 videobuf2_core v4l2_common videodev media ipv6
May 30 22:57:32 player-a8c4 kernel: [   78.079969] CPU: 0 PID: 2728 Comm: V4L2CaptureThre Not tainted 4.4.11-v7+ #888
May 30 22:57:32 player-a8c4 kernel: [   78.079973] Hardware name: BCM2709
May 30 22:57:32 player-a8c4 kernel: [   78.079996] [<80018724>] (unwind_backtrace) from [<80014058>] (show_stack+0x20/0x24)
May 30 22:57:32 player-a8c4 kernel: [   78.080009] [<80014058>] (show_stack) from [<803205a4>] (dump_stack+0xd4/0x118)
May 30 22:57:32 player-a8c4 kernel: [   78.080023] [<803205a4>] (dump_stack) from [<80025300>] (warn_slowpath_common+0x98/0xc8)
May 30 22:57:32 player-a8c4 kernel: [   78.080035] [<80025300>] (warn_slowpath_common) from [<800253ec>] (warn_slowpath_null+0x2c/0x34)
May 30 22:57:32 player-a8c4 kernel: [   78.080048] [<800253ec>] (warn_slowpath_null) from [<7f0b328c>] (__vb2_queue_cancel+0xfc/0x160 [videobuf2_core])
May 30 22:57:32 player-a8c4 kernel: [   78.080066] [<7f0b328c>] (__vb2_queue_cancel [videobuf2_core]) from [<7f0b3878>] (vb2_core_streamoff+0x44/0xac [videobuf2_core])
May 30 22:57:32 player-a8c4 kernel: [   78.080084] [<7f0b3878>] (vb2_core_streamoff [videobuf2_core]) from [<7f0bd5dc>] (vb2_streamoff+0x40/0x60 [videobuf2_v4l2])
May 30 22:57:32 player-a8c4 kernel: [   78.080100] [<7f0bd5dc>] (vb2_streamoff [videobuf2_v4l2]) from [<7f0bd64c>] (vb2_ioctl_streamoff+0x50/0x54 [videobuf2_v4l2])
May 30 22:57:32 player-a8c4 kernel: [   78.080125] [<7f0bd64c>] (vb2_ioctl_streamoff [videobuf2_v4l2]) from [<7f079d40>] (v4l_streamoff+0x28/0x2c [videodev])
May 30 22:57:32 player-a8c4 kernel: [   78.080153] [<7f079d40>] (v4l_streamoff [videodev]) from [<7f07de68>] (__video_do_ioctl+0x2a0/0x314 [videodev])
May 30 22:57:32 player-a8c4 kernel: [   78.080180] [<7f07de68>] (__video_do_ioctl [videodev]) from [<7f07d720>] (video_usercopy+0x188/0x60c [videodev])
May 30 22:57:32 player-a8c4 kernel: [   78.080207] [<7f07d720>] (video_usercopy [videodev]) from [<7f07dbc4>] (video_ioctl2+0x20/0x24 [videodev])
May 30 22:57:32 player-a8c4 kernel: [   78.080234] [<7f07dbc4>] (video_ioctl2 [videodev]) from [<7f078720>] (v4l2_ioctl+0xd4/0xe8 [videodev])
May 30 22:57:32 player-a8c4 kernel: [   78.080253] [<7f078720>] (v4l2_ioctl [videodev]) from [<80169288>] (do_vfs_ioctl+0x424/0x614)
May 30 22:57:32 player-a8c4 kernel: [   78.080263] [<80169288>] (do_vfs_ioctl) from [<801694bc>] (SyS_ioctl+0x44/0x6c)
May 30 22:57:32 player-a8c4 kernel: [   78.080273] [<801694bc>] (SyS_ioctl) from [<8000fb40>] (ret_fast_syscall+0x0/0x1c)
May 30 22:57:32 player-a8c4 kernel: [   78.080279] ---[ end trace f751a019b0ea7957 ]---

So it looks like width/height contraints won't impact the format.

Also, getUserMedia does usually looks good on OSX and Linux, is this 16x16 issue ARM specific?

6by9 commented 8 years ago

The app is still asking for 0x0, so it's still resetting to the default minimum of 16x16 which we know has issues. I am still trying to pin down why it doesn't work.

Most USB webcams only support specific resolutions. The chip used on Pi has a very powerful image processing block that should support cropping and scaling almost anything to anything, hence why it advertises 16x16 to 2592x1944 stepwise with a step of 2 in each dimension. Try using v4l2-ctl --list-formats-ext against your webcam on OSX or Linux desktop to see the list of resolutions supported - it'll be a pretty short list. I wouldn't say that was a strength of OSX / your Linux webcam.

TBH My simplest fix is going to be reset the minimum from 16x16 to 32x16 by bumping MIN_WIDTH at https://github.com/raspberrypi/linux/blob/rpi-4.4.y/drivers/media/platform/bcm2835/bcm2835-camera.c#L39. It'll work and I'll close this issue, but you'll get rubbish pictures - you need to work out how and why the framework you're using asks for such a silly resolution as that is not a kernel issue.

mgcrea commented 8 years ago

The exact same code with a Logitech C920 webcam plugged on the Pi (USB) does work and produce a very good looking image. Is there any chance that it only tries to grab a 0x0/16x16 frame to test if the camera actually works before switching to a better resolution?

Is there a way to get similar debug output than echo 2 | sudo tee /sys/module/bcm2835_v4l2/parameters/debug (frames asked, etc.) from the C920 Usb camera (to see a fully working handshake)?

EDIT: Just created an issue on Chromium bug tracker:

https://bugs.chromium.org/p/chromium/issues/detail?id=616007

EDIT 2: or maybe it just tries to grab the default resolution, the C920 first resolution in the list is 640x480, probably acting as default? Maybe a fix would bt to add a similar default/native discrete resolution for the bcm2835_v4l2 module?

EDIT 3: Also tried https://webrtchacks.github.io/WebRTC-Camera-Resolution/ (from this blog post), that polls several resolution, and it fails for every resolution using the camera module with a ConstraintNotSatisfiedError.

With the C920 plugged in:

v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
    Index       : 0
    Type        : Video Capture
    Pixel Format: 'YUYV'
    Name        : YUYV 4:2:2
        Size: Discrete 640x480
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 160x90
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 160x120
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 176x144
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 320x180
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 320x240
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 352x288
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 432x240
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 640x360
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 800x448
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 800x600
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 864x480
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 960x720
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1024x576
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1280x720
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1600x896
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1920x1080
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 2304x1296
            Interval: Discrete 0.500s (2.000 fps)
        Size: Discrete 2304x1536
            Interval: Discrete 0.500s (2.000 fps)

    Index       : 1
    Type        : Video Capture
    Pixel Format: 'H264' (compressed)
    Name        : H.264
        Size: Discrete 640x480
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 160x90
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 160x120
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 176x144
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 320x180
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 320x240
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 352x288
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 432x240
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 640x360
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 800x448
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 800x600
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 864x480
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 960x720
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1024x576
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1280x720
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1600x896
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1920x1080
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)

    Index       : 2
    Type        : Video Capture
    Pixel Format: 'MJPG' (compressed)
    Name        : Motion-JPEG
        Size: Discrete 640x480
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 160x90
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 160x120
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 176x144
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 320x180
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 320x240
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 352x288
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 432x240
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 640x360
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 800x448
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 800x600
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 864x480
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 960x720
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1024x576
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1280x720
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1600x896
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 1920x1080
            Interval: Discrete 0.033s (30.000 fps)
            Interval: Discrete 0.042s (24.000 fps)
            Interval: Discrete 0.050s (20.000 fps)
            Interval: Discrete 0.067s (15.000 fps)
            Interval: Discrete 0.100s (10.000 fps)
            Interval: Discrete 0.133s (7.500 fps)
            Interval: Discrete 0.200s (5.000 fps)

With the Camera module v2.1 plugged in:

ioctl: VIDIOC_ENUM_FMT
    Index       : 0
    Type        : Video Capture
    Pixel Format: 'YU12'
    Name        : Planar YUV 4:2:0
        Size: Stepwise 16x16 - 3280x2464 with step 2/2

    Index       : 1
    Type        : Video Capture
    Pixel Format: 'YUYV'
    Name        : YUYV 4:2:2
        Size: Stepwise 16x16 - 3280x2464 with step 2/2

    Index       : 2
    Type        : Video Capture
    Pixel Format: 'RGB3'
    Name        : 24-bit RGB 8-8-8
        Size: Stepwise 16x16 - 3280x2464 with step 2/2

    Index       : 3
    Type        : Video Capture
    Pixel Format: 'JPEG' (compressed)
    Name        : JFIF JPEG
        Size: Stepwise 16x16 - 3280x2464 with step 2/2

    Index       : 4
    Type        : Video Capture
    Pixel Format: 'H264' (compressed)
    Name        : H.264
        Size: Stepwise 16x16 - 3280x2464 with step 2/2

    Index       : 5
    Type        : Video Capture
    Pixel Format: 'MJPG' (compressed)
    Name        : Motion-JPEG
        Size: Stepwise 16x16 - 3280x2464 with step 2/2

    Index       : 6
    Type        : Video Capture
    Pixel Format: 'YVYU'
    Name        : YVYU 4:2:2
        Size: Stepwise 16x16 - 3280x2464 with step 2/2

    Index       : 7
    Type        : Video Capture
    Pixel Format: 'VYUY'
    Name        : VYUY 4:2:2
        Size: Stepwise 16x16 - 3280x2464 with step 2/2

    Index       : 8
    Type        : Video Capture
    Pixel Format: 'UYVY'
    Name        : UYVY 4:2:2
        Size: Stepwise 16x16 - 3280x2464 with step 2/2

    Index       : 9
    Type        : Video Capture
    Pixel Format: 'NV12'
    Name        : Y/CbCr 4:2:0
        Size: Stepwise 16x16 - 3280x2464 with step 2/2

    Index       : 10
    Type        : Video Capture
    Pixel Format: 'BGR3'
    Name        : 24-bit BGR 8-8-8
        Size: Stepwise 16x16 - 3280x2464 with step 2/2

    Index       : 11
    Type        : Video Capture
    Pixel Format: 'YV12'
    Name        : Planar YVU 4:2:0
        Size: Stepwise 16x16 - 3280x2464 with step 2/2

    Index       : 12
    Type        : Video Capture
    Pixel Format: 'NV21'
    Name        : Y/CrCb 4:2:0
        Size: Stepwise 16x16 - 3280x2464 with step 2/2

    Index       : 13
    Type        : Video Capture
    Pixel Format: 'BGR4'
    Name        : 32-bit BGRA/X 8-8-8-8
        Size: Stepwise 16x16 - 3280x2464 with step 2/2
6by9 commented 8 years ago

echo 0xFFFF | sudo tee > /sys/module/uvcvideo/parameters/trace

There is a default resolution set on the Pi - 1024x768 JPEG. Any call to VIDIOC_S_FMT is obliged to select a supported resolution. Docs are at http://hverkuil.home.xs4all.nl/spec/media.html#vidioc-g-fmt

When the application calls the VIDIOC_S_FMT ioctl with a pointer to a v4l2_format structure the driver checks and adjusts the parameters against hardware abilities.

Your app asked for 0x0. We checked against our capabilities and returned the closest match - 16x16.

It looks like the UVC (USB Video Class - almost all USB webcams) driver selects the default resolution advertised by the webcam - https://github.com/raspberrypi/linux/blob/rpi-4.4.y/drivers/media/usb/uvc/uvc_v4l2.c#L176 As you've seen, all the modes via UVC are listed as discrete modes, so it's easier to do that than search the list for the best match when you then have to balance off resolution vs aspect ratio vs frame rate. Personally I'd say that's not quite what the V4L2 spec says, but I'll defer to those doing the upstream development for UVC. There is no such concept as a default resolution supported by the device on Pi.

I think you must be missing something in your config. I doubt it's the latest code as it hasn't been updated in almost 2 years, but https://src.chromium.org/viewvc/chrome/trunk/src/media/video/capture/linux/video_capture_device_linux.cc line 258 is the call setting up the camera. Note that there are width & height parameters passed in to there. https://chromium.googlesource.com/chromium/src/+/master looks like a more up to date repo, but it's been rearranged compared to the other tree, and I'm sorry, I don't have time to dig into 3rd party apps to work out why/how they work with V4L2.

Google for "chromium getUserMedia resolution" seems to throw up a bundle of hits, including things like https://bugs.chromium.org/p/chromium/issues/detail?id=312928 where someone else was having issues trying to request a resolution. Have you followed all of the recommendations there?

I'm going to create a patch to shift the minimum res to being 32x32, otherwise I'm going to waste too much time on a calling app issue.

mgcrea commented 8 years ago

@6by9 Thank you for your time and help, I know it is precious. We will see what the chromium folks say on the ticket I've created.

I still think it's not an issue from my side (but indeed it may be a Chromium issue) as https://webrtchacks.github.io/WebRTC-Camera-Resolution/ works and properly set constraints for every combo (device/os+camera) I can try it with, it only fails with the Pi + Camera module.

For instance, here is the related output on the Pi for the C920: 

"Browser","Device","Res Name","Ratio","Ask","Actual","Status","deviceIndex","resIndex"
"chrome 51","HD Pro Webcam C920 (046d:082d)","4K(UHD)","16:9","3840x2160","0x0","fail: ConstraintNotSatisfiedError","0","0"
"chrome 51","HD Pro Webcam C920 (046d:082d)","1080p(FHD)","16:9","1920x1080","1920x1080","pass","0","1"
"chrome 51","HD Pro Webcam C920 (046d:082d)","UXGA","4:3","1600x1200","1600x1080","fail: mismatch","0","2"
"chrome 51","HD Pro Webcam C920 (046d:082d)","720p(HD)","16:9","1280x720","1280x720","pass","0","3"
"chrome 51","HD Pro Webcam C920 (046d:082d)","SVGA","4:3","800x600","800x600","pass","0","4"
"chrome 51","HD Pro Webcam C920 (046d:082d)","VGA","4:3","640x480","640x480","pass","0","5"
"chrome 51","HD Pro Webcam C920 (046d:082d)","360p(nHD)","16:9","640x360","640x360","pass","0","6"
"chrome 51","HD Pro Webcam C920 (046d:082d)","CIF","4:3","352x288","352x288","pass","0","7"
"chrome 51","HD Pro Webcam C920 (046d:082d)","QVGA","4:3","320x240","320x240","pass","0","8"
"chrome 51","HD Pro Webcam C920 (046d:082d)","QCIF","4:3","176x144","176x144","pass","0","9"
"chrome 51","HD Pro Webcam C920 (046d:082d)","QQVGA","4:3","160x120","160x120","pass","0","10"

And here it is for the Camera module:

"Browser","Device","Res Name","Ratio","Ask","Actual","Status","deviceIndex","resIndex"
"chrome 51","mmal service 16.1","4K(UHD)","16:9","3840x2160","0x0","fail: ConstraintNotSatisfiedError","0","0"
"chrome 51","mmal service 16.1","1080p(FHD)","16:9","1920x1080","0x0","fail: ConstraintNotSatisfiedError","0","1"
"chrome 51","mmal service 16.1","UXGA","4:3","1600x1200","0x0","fail: ConstraintNotSatisfiedError","0","2"
"chrome 51","mmal service 16.1","720p(HD)","16:9","1280x720","0x0","fail: ConstraintNotSatisfiedError","0","3"
"chrome 51","mmal service 16.1","SVGA","4:3","800x600","0x0","fail: ConstraintNotSatisfiedError","0","4"
"chrome 51","mmal service 16.1","VGA","4:3","640x480","0x0","fail: ConstraintNotSatisfiedError","0","5"
"chrome 51","mmal service 16.1","360p(nHD)","16:9","640x360","0x0","fail: ConstraintNotSatisfiedError","0","6"
"chrome 51","mmal service 16.1","CIF","4:3","352x288","0x0","fail: ConstraintNotSatisfiedError","0","7"
"chrome 51","mmal service 16.1","QVGA","4:3","320x240","0x0","fail: ConstraintNotSatisfiedError","0","8"
"chrome 51","mmal service 16.1","QCIF","4:3","176x144","0x0","fail: ConstraintNotSatisfiedError","0","9"
"chrome 51","mmal service 16.1","QQVGA","4:3","160x120","0x0","fail: ConstraintNotSatisfiedError","0","10"

https://bugs.chromium.org/p/chromium/issues/detail?id=312928 is related to multiple tab locking, I'm only starting a single electron app on a single url (nothing else is running on the device).

Thanks!

6by9 commented 8 years ago

https://bugs.chromium.org/p/chromium/issues/detail?id=312928 is related to multiple tab locking,

No, the first 11 comments are all related to setting the resolution. Multiple tabs are only mentioned in comment 14 causing the issue in comment 13.

Old commit, but may explain some things: https://chromium.googlesource.com/chromium/src/+/1577a17cdb0fe92f3e2dcd1ea2327ea814ca6549%5E!/

-    v4l2_frmsizeenum frame_size = {};
-    frame_size.pixel_format = v4l2_format.pixelformat;
-    for (; HANDLE_EINTR(ioctl(fd, VIDIOC_ENUM_FRAMESIZES, &frame_size)) == 0;
-         ++frame_size.index) {
-      if (frame_size.type == V4L2_FRMSIZE_TYPE_DISCRETE) {
-        supported_format.frame_size.SetSize(frame_size.discrete.width,
-                                            frame_size.discrete.height);
-      } else if (frame_size.type == V4L2_FRMSIZE_TYPE_STEPWISE ||
-                 frame_size.type == V4L2_FRMSIZE_TYPE_CONTINUOUS) {
-        // TODO(mcasas): see http://crbug.com/249953, support these devices.
-        NOTIMPLEMENTED();
-      }

That code was removed in the patch, but that implies STEPWISE is not supported. The bug report is still open, but that may be an oversight. https://chromium.googlesource.com/external/webrtc/trunk/webrtc/+/master/modules/video_capture/linux/device_info_linux.cc#242 implies that they may now implement the same sort of functionality using VIDIOC_TRY_FMT - last time I checked we passed v4l2-conformance testing on all of that , so I'd be surprised if that is going wrong. Is that the code that is running? Your client - go add some logging.

PR to increase minimum resolution to 32x32 has been merged. Either rebuild your own version of the kernel based on top of tree, or wait for an update to https://github.com/Hexxeh/rpi-firmware that can be collected with sudo rpi-update.

popcornmix commented 8 years ago

rpi-update kernel contains @6by9's increase of resolution to 32x32. It may be worth re-testing with this update.

mgcrea commented 8 years ago

Ok so I got it working with https://simpl.info/getusermedia/sources, but as predicted, I get a terrible image quality.

May 31 14:34:42 player-a8c4 kernel: [   42.765944] bcm2835-v4l2: Clipping/aligning 0x0 format 32315559
May 31 14:34:42 player-a8c4 kernel: [   42.765963] bcm2835-v4l2: Now 32x32 format 32315559
May 31 14:34:42 player-a8c4 kernel: [   42.765972] bcm2835-v4l2: vidioc_try_fmt_vid_cap: w 32 h 32 field 1 pfmt 0x32315559 bpl 32 sz_img 1536 colorspace 0x1 priv 0
May 31 14:34:42 player-a8c4 kernel: [   42.765978] bcm2835-v4l2: vid_cap - disconnect previous tunnel
May 31 14:34:42 player-a8c4 kernel: [   42.770502] bcm2835-v4l2: Set dev->capture.fmt 30323449, 32x32, stride 32, size 1536
May 31 14:34:42 player-a8c4 kernel: [   42.770541] bcm2835-v4l2: Set fps range to 300000/10000 to 300000/10000
May 31 14:34:42 player-a8c4 kernel: [   42.770748] bcm2835-v4l2: queue_setup: dev:b5b3c000
May 31 14:34:42 player-a8c4 kernel: [   42.770854] bcm2835-v4l2: buffer_prepare: dev:b5b3c000
May 31 14:34:42 player-a8c4 kernel: [   42.770887] bcm2835-v4l2: buffer_prepare: dev:b5b3c000
May 31 14:34:42 player-a8c4 kernel: [   42.770912] bcm2835-v4l2: buffer_prepare: dev:b5b3c000
May 31 14:34:42 player-a8c4 kernel: [   42.770935] bcm2835-v4l2: buffer_prepare: dev:b5b3c000
May 31 14:34:42 player-a8c4 kernel: [   42.770947] bcm2835-v4l2: buffer_queue: dev:b5b3c000 buf:9963b400
May 31 14:34:42 player-a8c4 kernel: [   42.770953] bcm2835-v4l2: buffer_queue: dev:b5b3c000 buf:9963b200
May 31 14:34:42 player-a8c4 kernel: [   42.770958] bcm2835-v4l2: buffer_queue: dev:b5b3c000 buf:9963ac00
May 31 14:34:42 player-a8c4 kernel: [   42.770964] bcm2835-v4l2: buffer_queue: dev:b5b3c000 buf:9963a000
May 31 14:34:42 player-a8c4 kernel: [   42.770969] bcm2835-v4l2: start_streaming: dev:b5b3c000
May 31 14:34:42 player-a8c4 kernel: [   43.124456] bcm2835-v4l2: enabled camera (refcount 1)
May 31 14:34:42 player-a8c4 kernel: [   43.427878] bcm2835-v4l2: Start time 47686050 size 8
May 31 14:34:42 player-a8c4 kernel: [   43.439083] bcm2835-v4l2: buffer_cb: status:0, buf:9963b400, length:1536, flags 4, pts 47567210
May 31 14:34:42 player-a8c4 kernel: [   43.439099] bcm2835-v4l2: Convert start time 43.400196 and 47686050 with offset 47567210 to -140462568.832972
May 31 14:34:42 player-a8c4 kernel: [   43.439698] bcm2835-v4l2: buffer_prepare: dev:b5b3c000
May 31 14:34:42 player-a8c4 kernel: [   43.439711] bcm2835-v4l2: buffer_queue: dev:b5b3c000 buf:9963b400
May 31 14:34:42 player-a8c4 kernel: [   43.471692] bcm2835-v4l2: buffer_cb: status:0, buf:9963b200, length:1536, flags 4, pts 47700519
May 31 14:34:42 player-a8c4 kernel: [   43.471706] bcm2835-v4l2: Convert start time 43.400196 and 47686050 with offset 47700519 to 43.414665
[...]

Still no luck with custom resolutions: https://webrtchacks.github.io/WebRTC-Camera-Resolution/

"Browser","Device","Res Name","Ratio","Ask","Actual","Status","deviceIndex","resIndex"
"chrome 51","mmal service 16.1","4K(UHD)","16:9","3840x2160","0x0","fail: ConstraintNotSatisfiedError","0","0"
"chrome 51","mmal service 16.1","1080p(FHD)","16:9","1920x1080","0x0","fail: ConstraintNotSatisfiedError","0","1"
"chrome 51","mmal service 16.1","UXGA","4:3","1600x1200","0x0","fail: ConstraintNotSatisfiedError","0","2"
"chrome 51","mmal service 16.1","720p(HD)","16:9","1280x720","0x0","fail: ConstraintNotSatisfiedError","0","3"
"chrome 51","mmal service 16.1","SVGA","4:3","800x600","0x0","fail: ConstraintNotSatisfiedError","0","4"
"chrome 51","mmal service 16.1","VGA","4:3","640x480","0x0","fail: ConstraintNotSatisfiedError","0","5"
"chrome 51","mmal service 16.1","360p(nHD)","16:9","640x360","0x0","fail: ConstraintNotSatisfiedError","0","6"
"chrome 51","mmal service 16.1","CIF","4:3","352x288","0x0","fail: ConstraintNotSatisfiedError","0","7"
"chrome 51","mmal service 16.1","QVGA","4:3","320x240","0x0","fail: ConstraintNotSatisfiedError","0","8"
"chrome 51","mmal service 16.1","QCIF","4:3","176x144","0x0","fail: ConstraintNotSatisfiedError","0","9"
"chrome 51","mmal service 16.1","QQVGA","4:3","160x120","0x0","fail: ConstraintNotSatisfiedError","0","10"

I'm running an up-to-date Electron v1.2.0 based on Chromium 51.0.2704.54. So it looks like the code actually running might be:

EDIT 1: Just found that I can simply add the --enable-logging flag

https://simpl.info/getusermedia/sources outputs:

[25420:0531/150505:INFO:CONSOLE(0)] "MediaStreamTrack.getSources is deprecated. See https://www.chromestatus.com/feature/4765305641369600 for more details.", source:  (0)
[25420:0531/150505:ERROR:video_capture_device_factory_linux.cc(108)] Not implemented reached in void media::GetSupportedFormatsForV4L2BufferType(int, media::VideoCaptureFormats *)
[25420:0531/150505:ERROR:video_capture_device_factory_linux.cc(108)] Not implemented reached in void media::GetSupportedFormatsForV4L2BufferType(int, media::VideoCaptureFormats *)
[25420:0531/150505:ERROR:video_capture_device_factory_linux.cc(108)] Not implemented reached in void media::GetSupportedFormatsForV4L2BufferType(int, media::VideoCaptureFormats *)
[25420:0531/150505:ERROR:video_capture_device_factory_linux.cc(108)] Not implemented reached in void media::GetSupportedFormatsForV4L2BufferType(int, media::VideoCaptureFormats *)
[25420:0531/150505:ERROR:video_capture_device_factory_linux.cc(108)] Not implemented reached in void media::GetSupportedFormatsForV4L2BufferType(int, media::VideoCaptureFormats *)
[25420:0531/150505:ERROR:video_capture_device_factory_linux.cc(108)] Not implemented reached in void media::GetSupportedFormatsForV4L2BufferType(int, media::VideoCaptureFormats *)
[25420:0531/150505:WARNING:media_stream_manager.cc(995)] Invalid optional capture ID =
[25420:0531/150505:WARNING:media_stream_manager.cc(995)] Invalid optional capture ID =

Which links to this file: https://chromium.googlesource.com/chromium/src/+/51.0.2704.54/media/capture/video/linux/video_capture_device_factory_linux.cc

That confirms your suspicion:

      } else if (frame_size.type == V4L2_FRMSIZE_TYPE_STEPWISE ||
                 frame_size.type == V4L2_FRMSIZE_TYPE_CONTINUOUS) {
        // TODO(mcasas): see http://crbug.com/249953, support these devices.
        NOTIMPLEMENTED();
      }

So http://crbug.com/249953 it is. Any chance that the Raspberry Pi could provide both discrete and stepwise resolutions?

6by9 commented 8 years ago

So http://crbug.com/249953 it is. Any chance that the Raspberry Pi could provide both discrete and stepwise resolutions?

No, they're mutually exclusive, reported back via a union - see http://hverkuil.home.xs4all.nl/spec/media.html#vidioc-enum-framesizes and http://hverkuil.home.xs4all.nl/spec/media.html#v4l2-frmsizeenum. Admittedly there is a hack already in place because GStreamer didn't handle STEPWISE correctly either, but behaves if VIDIOC_ENUM_FRAMESIZES is not supported. There could be a second hack added for webRTC, but you then start asking the question of which resolutions do you advertise - it could be a long list, and multiply that by all the supported pixel formats and it becomes a nonsense.

It looks like they have already fixed it to some degree with the updated code I linked to, and it makes a call over which resolutions it wants to query for. Are you feeling brave and fancy compiling top of tree?! Unless your version has long term support, I suspect that is all the Chromium guys are going to direct you to do anyway.

6by9 commented 8 years ago

BTW You're going to hit an unsupported at https://chromium.googlesource.com/chromium/src/+/51.0.2704.54/media/capture/video/linux/video_capture_device_factory_linux.cc#71 too, for frame intervals (aka frame rate). The Pi camera supports a V4L2_FRMIVAL_TYPE_CONTINUOUS range from 1/90 to 1/1 second frame interval, but the webRTC code doesn't support that enum.

mgcrea commented 8 years ago

There could be a second hack added for webRTC, but you then start asking the question of which resolutions do you advertise - it could be a long list, and multiply that by all the supported pixel formats and it becomes a nonsense.

Being excited to play with webRTC on the Pi, I obviously really like this idea. I guess we could advertise relatively standard resolutions in the YUYV 4:2:2 format such as:

"VGA","4:3","640x480"
"SVGA","4:3","800x600"
"720p(HD)","16:9","1280x720"
"UXGA","4:3","1600x1200"
"1080p(FHD)","16:9","1920x1080"

That would fix 99% of the webRTC use cases.

It looks like they have already fixed it to some degree with the updated code I linked to, and it makes a call over which resolutions it wants to query for. Are you feeling brave and fancy compiling top of tree?!

I don't really get the relation between chromium's video_capture_device_factory_linux.cc and the code you linked webrtc's device_info_linux.cc

Though I'd gladly spend some time digging into it, I am a bit clueless regarding the step needed to compile top-of-tree of external/webrtc into chromium/src.

6by9 commented 8 years ago

YUYV is not the ideal format on Pi. I'd recommend you use V4L2_PIX_FMT_YUV420. If you really have to, then that could be done in the driver, but I'd suggest you look at fixing Chromium first (it's the correct way to fix it).

My bad on directory. I'd followed https://www.chromium.org/developers/how-tos/get-the-code to grab the depot-tools and then fetch chromium and it fetched 15GB of stuff. A grep for VIDIOC_S_FMT (or was it a Google) had shown up the webrtc stuff first, but media/capture/video/linux/video_capture_device_factory_linux.cc looks like it is the correct code.

So first step is to get ToT building, expecting it still to fail. Then lift the code webRTC doing VIDIOC_TRY_FMT on your desired formats, and storing those into supported_formats if they succeed. You'll need to do something in GetFrameRateList to handle getting continuous frame rate values as well (again go for some preferred values). Push upstream for review and get your name in lights as a contributor to the project!

6by9 commented 7 years ago

A comment from having this issue discussed on the forums by another user (https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=159941)

I had the daft thought of what happens if you use the GStreamer hack (don't advertise any resolutions) with Chromium. Apparently it works, although I have no idea what format or resolution it is actually asking for - it'd be nice to check. sudo modprobe bcm2835-v4l2 gst_v4l2src_is_broken=1 loads the V4L2 driver with the GST hack enabled.

JamesH65 commented 7 years ago

@6by9 I suspect this can be closed?

6by9 commented 7 years ago

Yes, close. It's a bug in Chromium. There is a workaround in setting bcm2835-v4l2 gst_v4l2src_is_broken=1 (at least until we get the upstream V4L2 driver merged where they've decided to remove the workaround as GStreamer have fixed their bug)

andi34 commented 2 years ago

Hey together! Sorry that ATM I can't give much information but i am wondering if chromium does not detect the Pi Camera as device camera since some months because inside drivers/media/platform/bcm2835/bcm2835-unicam.c the MIN_WIDTH and MIN_HEIGHT are set to 16 inside 5.10y.

raspistill on buster based Pi OS do work which why I think it could be an issue with chromium and our camera driver.

Can't tell about Pi OS based on bullseye.

I am going to check the dmesg once I have enough time. Maybe in the meantime someone can tell if the issue appears too for him/her?

Apteryks commented 1 year ago

Hi!

Sorry if I got lost somewhere in this longish thread, but why is this issue closed? It seems it was closed as a duplicate of #1499 in https://github.com/raspberrypi/linux/issues/1498#issuecomment-222301866, but #1499 was also closed as a duplicate of this one.

In any case, the issue still exists in Bullseye (Chromium or Electron doesn't detect the Raspberry Pi v1 camera module), so it'd be good to keep this open, linking to the currently open corresponding upstream bug: https://bugs.chromium.org/p/chromium/issues/detail?id=1406830.

Thanks!

andi34 commented 1 year ago

Heres the additions the pi foundation added to Chromium in the past, afaik it's now used from the official debian Sources

https://github.com/RPi-Distro/chromium-browser/files/7829777/camera.txt

andi34 commented 1 year ago

See https://github.com/RPi-Distro/chromium-browser/issues/23 for reference

6by9 commented 1 year ago

It's closed because that issue was solved - Chromium didn't support V4L2 camera drivers that advertised the resolutions as V4L2_FRMSIZE_TYPE_STEPWISE or V4L2_FRMSIZE_TYPE_CONTINUOUS.

The issue you're probably hitting now is that Chromium doesn't support libcamera, and the default camera stack for Raspberry Pi OS is now that. I believe the preferred route for supporting libcamera within Chromium is going to be via Pipewire, but I could be wrong.

Apteryks commented 1 year ago

I'm hitting the issue even using the legacy camera stack; which shouldn't happen if Chromium is now patched as above, if I understand correctly. This is using a Raspberry Pi 3 B and a camera module v1. I've detailed my findings and workarounds in https://forums.raspberrypi.com/viewtopic.php?t=351986.

Apteryks commented 1 year ago

@andi34 it doesn't appear that this custom patch has made its way in either Chromium or the Debian package of it; see the patches used in: http://deb.debian.org/debian/pool/main/c/chromium/chromium_113.0.5672.126-1.debian.tar.xz, which is for the Debian chromium package in Bookworm.

Looking at the chromium source http://deb.debian.org/debian/pool/main/c/chromium/chromium_113.0.5672.126.orig.tar.xz shows the patch hasn't been applied there either. Unless I've missed something, it means this bug is still relevant. Could this issue please be reopened?

I've also requested the upstream bug https://bugs.chromium.org/p/chromium/issues/detail?id=249953#c25 to be reopened, as it's been archived without resolution while the source code still refers to it, calling NOTIMPLEMENTED_LOG_ONCE(); at the place the STEPWISE frame type camera should be handled.

andi34 commented 1 year ago

The patch was / diff was shared by @jc-kynesim while I asked for a solution. And yes, it is not added to the official chromium sources. I was told the pi foundation used it's own sources and later shwitched to the mainline package.

andi34 commented 1 year ago

Besides that it's also still an issue on a 32bit PiOS to use the v4l2loopback module as it fails because of the mixed arch of the kernel and Userland.... https://github.com/raspberrypi/linux/issues/5410

Apteryks commented 1 year ago

Related, see an account of my struggle/hacky workarounds using v4l2loopback (I had to switch to 32 bits kernel/userland to avoid the above issue you linked) here: https://forums.raspberrypi.com/viewtopic.php?p=2109999#p2109999