Please upstream modules into mainline kernel

[bk2204]

I recently discovered that my ThinkPad X1 Carbon needs an out-of-tree driver (this one) for my camera. That seems atypical for Intel hardware, which is typically well supported upstream.

I've tried with both Debian sid's 5.18.0-2-amd64 and Ubuntu jammy's 5.15, but the driver is included in neither distro, probably because it isn't upstream. Since this hardware is considered important for most people who use an Alder Lake laptop, could you consider upstreaming this driver (and any dependent pieces) into the mainline kernel relatively soon, even if only in the staging area?

I did try to build this myself with DKMS, but due to #13 (both the missing header and then the modpost error), it's not possible to do so, so there is presently no way to make this hardware work on Debian.

[dennis-pries]

Fedora (and I think a lot of other distos) also will not include them, if not upstreamed.

Please do it.

[wbolster]

the new Dell XPS 13 Plus (9320) also requires this driver for the webcam, but building it fails on linux kernel 5.18.7 (arch linux) due to the issues in #13 (missing headers, and unresolved symbols after manually copying headers)

[naps62]

I also have a Dell XPS 13 plus, and just stumbled upon this issue. Also failed to build. I'm not very profficient in kernel module compilation. Is there a way to get this working at the moment? I'm using Manjaro

[K-MeLeOn]

Dell XPS 13 Plus too, building error on modpost with message 'undefined!' on ov02c10.ko and ov01a10.ko

[paulmenzel]

I notified the Linux upstream folks about the issue. The current recommendation is to not buy such devices. From Greg KH:

On Thu, Aug 11, 2022 at 04:54:53PM +0300, Laurent Pinchart wrote:

For the time being, I agree with your recommendation to not buy these devices if you care about camera support.

I second this, don't buy these devices if the vendor is not willing to get their drivers upstreamed properly.

It also sounds like, that it won’t be easily be solved.

Those of you who can, I can only suggest to return the device to not further support vendors screwing up Linux support in this case, and save time fiddling with Linux driver issues.

No idea, who dropped the ball here, but the ChromiumOS/ChromeOS and Ubuntu/Canonical/Dell/Lenovo should have started the discussion with the upstream folks much earlier. It’s a very sad situation for the GNU/Linux ecosystem.

[kbingham]

We can certainly help over at https://libcamera.org if anyone from Intel wants to contact us.

[pinchartl]

@paulmenzel I share your disappointment, but I'd like to focus on how we could improve the situation instead of generating bad PR, as I'm sure the latter doesn't need my help :-)

I won't comment on what happened at Google, Canonical, Dell or Lenovo, as only them have a full picture. All I can say is that it's fairly common for discussions to start in private to decide how to support new product, especially before they are released. I have no reason to believe those discussions didn't start, but they obviously didn't bear fruits so far. The problem isn't easy to solve, and I do understand how development teams can be pressured with product deadlines. Still, this may explain the situation, but not excuse it.

Another point I'd like to make is that we have received technical support from multiple Intel developers when developing IPU3 support in libcamera. There are people with goodwill in their teams, I don't think it would be fair blaming all the developers for the lack of a high-level upstreaming strategy.

We won't be able to fix the mistakes that were made without a clear commitment from Intel, they need to engage directly with the Linux kernel and libcamera upstream developers. Given the amount of work that lies ahead, this has to happen now. My door is fully open.

[Thesola10]

hey, in the mean time I managed to create a patch for pure DKMS building in my AUR package intel-ipu6-dkms-git

[Vascro]

Doesn't appear there are any new updates regarding this ipu6 mipi camera issue. I just got my new XPS 9320 in the mail and noticed this issue immediately. Hopefully Intel fixes this ASAP.

[paulmenzel]

I can only recommend to return the system to not support such products. Otherwise vendors won’t change as long as they can sell such devices.

Additionally, Jonathan Corbet summarized the topic in the LWN article The growing image-processor unpleasantness.

[naps62]

I'm also stuck with a Dell XPS 13 with this issue. don't have a chance to return it unfortunately, although I do agree returning sends the more forceful message to vendors.

FYI @Thesola10 I tried your kernel module (although I'm not 100% sure what it would achieve) on my Manjaro installation. The boot hanged for a few minutes but ended up booting in the end. Stilll no working though, so I removed it again to fix the boot process

[Thesola10]

The lengthened boot process is due to including the intel_vsc drivers into the initramfs. Those can be excluded and they'll load after boot. Otherwise the driver works with icamerasrc on GStreamer so you can try installing that as well

[cjechlitschek]

This might be already known, but I still want to mention it. Canonical provides prebuilt packages for some Dell systems that enable the camera: https://wiki.ubuntu.com/Dell

[netllama]

@cjechlitschek those packages are completely useless if someone isn't running a Debian based distro. Also, see the previous comments which explain why those packages are flawed.

[corebots]

I also miss the webcam under Fedora, pleas fix who can!

[githubtefo]

This is crazy, I just bought the XPS 13 Plus because it was certified by Ubuntu :weary: and even under a Debian based distro, the drivers are not working on 5.17.0-1018-oem kernel.

[naps62]

@githubtefo For what it's worth, I've been running Ubuntu smoothly for a while now, with snapd and all the bloatware removed I had already bought the laptop, after a 3-month wait, and shipped from US to Europe. so it's not like I had much of a choice

will happily switch back to Arch once possible. But all things considered, I'm quite happy with the performance of the laptop, including the webcam

[karolszk]

Hello, I would like also point out the need for camera drivers for the newest Lenovo Carbon X10 laptop. For 6 version of Linux kernel where are compilable and still lacking in the upstream.

[simisimis]

for my new job I was picking from Thinkpad X1 gen10 and Dell XPS Plus. Was afraid that I made a wrong choice. But from what you wrote I see I would have ended up in this thread regardless 😅

[corebots]

Hi, I've picked up the XPS 13 Plus 5 months ago, been using it with Fedora since then. Mostly good, except the webcam still not working after all that time :(

Please add the webcam drivers to the mainline kernel since it's a bummer needing to use an external usb webcam with such a great and expensive machine. Thanks!

[dmipx]

We are from RealSense working on a "webcam" support for IPU6. This will support MIPI connected camera on IPU6 to act as "USB UVC", provide access to camera features through video node. First of all, we want to support RealSense D457 MIPI camera on IPU6 platform.

The work is in progress - we estimate work done by endo of the year, you can observe PR's update. https://github.com/intel/ipu6-drivers/pull/73 My repo with updates: https://github.com/dmipx/ipu6-drivers/tree/iotg_ipu6_isys_video_ctrl

[netllama]

@dmipx that sounds like great news. Hopefully you can improve the README as well, so that there are clear build instructions, instead of the current outdated confusing mess.

[dmipx]

@netllama AFAIK the iotg_ipu6 branch is heavily developed so you can sync with it and see if there is any issue with build steps. I'm not sure about all supported camera drivers but i know that IPU6 driver can be built without issues.

[pinchartl]

We are from RealSense working on a "webcam" support for IPU6. This will support MIPI connected camera on IPU6 to act as "USB UVC", provide access to camera features through video node. First of all, we want to support RealSense D457 MIPI camera on IPU6 platform.

The work is in progress - we estimate work done by endo of the year, you can observe PR's update. #73 My repo with updates: https://github.com/dmipx/ipu6-drivers/tree/iotg_ipu6_isys_video_ctrl

Are you planning to upstream this work, or just merge it in the out-of-tree IPU6 driver ?

[dmipx]

For now we have dispute about it with ipu6 team so we started to implement it as out-of-tree. RealSense, as usual, will provide debian package for ubuntu distros with DKMS supporting d457 cameras. If it will be demanded i will try to support laptops mipi cameras but for now - I have no hardware to work on it. Let's see how it goes. In long term we want to upstream but this should be community demand.

[paulmenzel]

In long term we want to upstream but this should be community demand.

Please contact the Linux (subsystem) kernel mailing list with the involved developers in Cc notifying them about your efforts. This GitHub issue is not the right forum for this.

[MorganMLGman]

Hi, any updates regarding this topic? Is it possible to have this included in mainline kernel or people with alder lake laptops will be left with not working webcam on linux?

[chromer030]

Please upstream the module , people are suffering from lack of support.

[SmokinCaterpillar]

Hey is there any update? I bought a new X1Gen10 and, working in a remote setting without a camera, this machine is a bit of a very expensive paperweight.

[paulmenzel]

@SmokinCaterpillar, [comment from Lenovo’s Mark Pearson]():

100% agreed - don't buy the platforms (X1 Carbon G10, X1 Yoga G7, X1 Nano G2) with the MIPI camera (called 'computer vision' on the Lenovo site) if you plan to run Linux. We are disabling that config on our Linux preload.

Though on their Web site they still list Ubuntu and Fedora as supported:

Betriebssystem:

• Bis zu Windows 11 Pro
• Linux® (Ubuntu)
• Linux (Fedora)

[dennis-pries]

With Ubuntu 22.04 the Camera does work, but it's a PITA implementation. You need a daemon that copies the video stream from the driver into a v4l loopback device (v4l2-relayd) and a gstreamer plugin to get it working... So sadly, simply embedding the driver into the kernel isn't enough to get it working...

And if it works, there is a high cpu load when using it, because it has to copy the video stream from one place to another.

Avoid this hardware if you want a working internal webcam and Linux.

[SmokinCaterpillar]

Avoid this hardware is hardly possible if you a) want a screen resolution higher than 1920 x 1200 or b) you need 32GB or more RAM (hello Pycharm :-D), as any X1 config that offers both has the MiPi camera with it.

So the only option would be to move away completely from ThinkPads and look elsewhere. I mean my coworkers keep nagging me to switch to Apple :-/

I'd rather say if Intel wants to be respected in the Developer community, they should get this working properly for all Linux Distros asap.

[paulmenzel]

@SmokinCaterpillar, not buying is the only option you have as a consumer. Right now, Lenovo (and Dell, …) and therefore Intel made money from people buying these devices despite the shortcoming. So why should they change their behavior?

Anyway, this is off-topic. As you can see from the issue discussion, nobody official responds here, so it’s just for venting. If you like to have an update, please contact the Lenovo support to raise this issue internally at Lenovo and reply to the mailing list discussion, asking for an update there.

(For the netiquette, please import the archive from lore.kernel.org, and do not top post but use interleaved style. LKML also only excepts text/plain messages with no html/plain.)

[SmokinCaterpillar]

I mean from now on until this is properly sorted out I am going to advice our employees against buying ThinkPads or any other setup with Intel MiPi cameras. We are just a small startup, so it's just going to be a couple of thousands for a handful of notebooks, but hey every little bit counts.

I also raised the issue with the Lenovo support, I wrote into a couple of forum threads in the Lenovo forum as well as the Intel forum. So I am hopefully already a bit annoying to them to get a tiny bit of movement.

I mean I already would be happy if someone from Intel would just give a proper update about the matter that they are actively working on it and come up with a solution in a reasonable amount of time. So hey, anyone here willing to do that?

[dmipx]

Hi. I'm working for Intel RealSense. There is a hope for MiPi cameras to work on Intel platforms. Currently i have a solution for RealSense MiPi cameras to be a native V4L2 and, hopefully, when i will get my hands to one of the laptops i can integrate these changes to work with internal webcam. For the time being, the only one Engineer that can resolve this issue have no access to hardware to work on.

[dmipx]

@dennis-pries Can you clarify what webcam sensor driver your laptop uses? I can try to build modules "blindly", without verification.

[paulmenzel]

@dmipx, thank you for the update. Although it sounds strange, that you couldn’t get access to such a device, I guess Dell, Lenovo and Canonical can give the developers such a device. @mrhpearson?

[mkobel]

I have a X1 Gen 10 running on Arch Linux with "regular" kernel 6.2.8.arch1-1 and optionally the LTS kernel package 6.1.21-1. I'm no kernel developer but open to support testing/debugging.

[pinchartl]

@dmipx Are you sure you're not confusing different things?

Supporting the IPU6 properly will require upstreaming a driver for the IPU6 input system (more or less the CSI-2 receivers), a driver for the IPU6 processing system (the memory-to-memory ISP), glue code to work around deficient ACPI integration, and depending on the platform, sensor drivers and drivers for the external Intel visual sensing controller (IVSC). The IVSC, as far as I understand, is mostly present in platforms made to support Windows Hello and may thus not be present on machines designed for Linux (which is all for the best, as it's a man-in-the-middle attack vector in the form or a chip sitting between the camera sensor and the Intel SoC, and running a fully closed firmware, integrated with the closed-source management engine).

There's an ongoing effort to upstream bare minimum support for the IVSC in the form of a driver that configures the IVSC in bypass mode (see https://lore.kernel.org/linux-media/1680226031-23163-1-git-send-email-wentong.wu@intel.com). For machine integrating an IVSC this driver is required to route the video stream from the camera sensor to the IPU6, but that's all it does, so you won't get a working camera yet.

I can't speak for Intel when it comes to a schedule to upstream drivers for the input system and processing system. The former should be simpler to upstream (even if technically more complex), and will provide the ability to capture raw images, which is better than nothing. I have good hopes that a driver will be submitted at some point this year. For the processing system, the situation is more complex, and a driver alone won't give you a camera. Image processing algorithms that control the IPU6 processing system will need to be developed and integrated in the libcamera project (https://libcamera.org/), which is another component that will require significant development effort.

[SmokinCaterpillar]

@dennis-pries Can you clarify what webcam sensor driver your laptop uses? I can try to build modules "blindly", without verification.

I also got the camera working with all four repositories linked in the README and this CPU intense gstreamer workaround.

ls -ld /sys/class/video4linux/video0/device/driver returns

lrwxrwxrwx 1 root root 0 Apr 1 11:53 /sys/class/video4linux/video0/device/driver -> ../../../../bus/intel-ipu6-bus/drivers/intel-ipu6-isys

if that helps.

And according to the Lenovo support page the camera is just described as "Camera,FHD,RGB+IR,MIPI,UF,Azw"

[dmipx]

Let's try to figure out with lsmod | grep ipu

[dmipx]

I have a X1 Gen 10 running on Arch Linux with "regular" kernel 6.2.8.arch1-1 and optionally the LTS kernel package 6.1.21-1.

I'm no kernel developer but open to support testing/debugging.

Looks like you should have "ov2740" imaging sensor. Can you confirm?

[karolszk]

Hello, I have Thinkpad Carbon Gen10 too and says as you suspecting: ~$ lsmod | grep ipu intel_ipu6_isys 139264 0 videobuf2_dma_contig 20480 1 intel_ipu6_isys videobuf2_v4l2 36864 2 intel_ipu6_isys,uvcvideo videobuf2_common 73728 6 videobuf2_vmalloc,videobuf2_dma_contig,videobuf2_v4l2,intel_ipu6_isys,uvcvideo,videobuf2_memops v4l2_fwnode 32768 2 intel_ipu6_isys,ov2740 intel_ipu6_psys 98304 0 v4l2_async 32768 3 v4l2_fwnode,intel_ipu6_isys,ov2740 intel_ipu6 106496 2 intel_ipu6_isys,intel_ipu6_psys videodev 294912 6 v4l2_async,videobuf2_v4l2,intel_ipu6_isys,uvcvideo,videobuf2_common,ov2740 mc 77824 8 v4l2_async,videodev,snd_usb_audio,videobuf2_v4l2,intel_ipu6_isys,uvcvideo,videobuf2_common,ov2740

[mrhpearson]

@dmipx, thank you for the update. Although it sounds strange, that you couldn’t get access to such a device, I guess Dell, Lenovo and Canonical can give the developers such a device. @mrhpearson?

Hi - we (Lenovo) have given platforms for the Intel camera team to work on - they have last years and this years platforms. To my knowledge we've given them everything they asked for. I was actually a little bit surprised to see 'no access to hardware to work on' posted from an Intel engineer in a public forum.

@dmipx - Let me know what I'm missing - I have not had any requests from Intel for more platforms. Are you working with the Intel camera team on this? If there are things we should be doing to accelerate getting support upstream for this device please let me know (probably privately would be best). I will happily support anything that gets this project moving quicker!

Thanks Mark

[SmokinCaterpillar]

Let's try to figure out with lsmod | grep ipu

Same, same

$ lsmod | grep ipu

gives

intel_ipu6_isys 143360 0 videobuf2_dma_contig 24576 1 intel_ipu6_isys videobuf2_v4l2 32768 1 intel_ipu6_isys videobuf2_common 81920 4 videobuf2_dma_contig,videobuf2_v4l2,intel_ipu6_isys,videobuf2_memops intel_ipu6_psys 98304 0 intel_ipu6 106496 2 intel_ipu6_isys,intel_ipu6_psys v4l2_fwnode 32768 2 intel_ipu6_isys,ov2740 v4l2_async 28672 3 v4l2_fwnode,intel_ipu6_isys,ov2740 videodev 274432 6 v4l2_async,videobuf2_v4l2,v4l2loopback,intel_ipu6_isys,videobuf2_common,ov2740 mc 65536 6 v4l2_async,videodev,videobuf2_v4l2,intel_ipu6_isys,videobuf2_common,ov2740

[dmipx]

@mrhpearson thank you for quick response. Intel is a big company, I am working for RealSense team and we have d457 mipi camera fully integrated with ipu6 working like charm as regular webcam. We have hard time with ipu6 team - working on "impossible" features with great success.

[dmipx]

@SmokinCaterpillar @karolszk can someone of you attach text file with following command:

media-ctl --print-dot

I can try to estimate work to be done.

[karolszk]

Hi @dmipx

$ media-ctl --print-dot Failed to enumerate /dev/media0 (-2)

[mkobel]

lsmod | grep ipu

Yes, I can confirm ov2740:

intel_ipu6_isys       172032  0
videobuf2_dma_contig    32768  1 intel_ipu6_isys
videobuf2_v4l2         40960  1 intel_ipu6_isys
videobuf2_common       90112  4 videobuf2_dma_contig,videobuf2_v4l2,intel_ipu6_isys,videobuf2_memops
intel_ipu6_psys       126976  0
v4l2_fwnode            32768  2 intel_ipu6_isys,ov2740
intel_ipu6            143360  2 intel_ipu6_isys,intel_ipu6_psys
v4l2_async             32768  3 v4l2_fwnode,intel_ipu6_isys,ov2740
videodev              352256  4 v4l2_async,videobuf2_v4l2,intel_ipu6_isys,ov2740
mc                     81920  6 v4l2_async,videodev,videobuf2_v4l2,intel_ipu6_isys,videobuf2_common,ov2740

[mkobel]

Lenovo X1 10Gen / Linux 6.2.9-arch1-1 / OV2740: $ media-ctl --print-dot

digraph board {
    rankdir=TB
    n00000001 [label="{{<port0> 0} | Intel IPU6 CSI-2 0\n/dev/v4l-subdev0 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000001:port1 -> n00000019:port0 [style=dashed]
    n00000004 [label="{{<port0> 0} | Intel IPU6 CSI-2 1\n/dev/v4l-subdev1 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000004:port1 -> n00000019:port0 [style=dashed]
    n00000007 [label="{{<port0> 0} | Intel IPU6 CSI-2 2\n/dev/v4l-subdev2 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000007:port1 -> n00000019:port0 [style=dashed]
    n0000000a [label="{{<port0> 0} | Intel IPU6 CSI-2 3\n/dev/v4l-subdev3 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n0000000a:port1 -> n00000019:port0 [style=dashed]
    n0000000d [label="{{<port0> 0} | Intel IPU6 CSI-2 4\n/dev/v4l-subdev4 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n0000000d:port1 -> n00000019:port0 [style=dashed]
    n00000010 [label="{{<port0> 0} | Intel IPU6 CSI-2 5\n/dev/v4l-subdev5 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000010:port1 -> n00000019:port0 [style=dashed]
    n00000013 [label="{{<port0> 0} | Intel IPU6 CSI-2 6\n/dev/v4l-subdev6 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000013:port1 -> n00000019:port0 [style=dashed]
    n00000016 [label="{{<port0> 0} | Intel IPU6 CSI-2 7\n/dev/v4l-subdev7 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000016:port1 -> n00000019:port0 [style=dashed]
    n00000019 [label="{{<port0> 0} | Intel IPU6 CSI2 BE SOC 0\n/dev/v4l-subdev8 | {<port1> 1}}", shape=Mrecord, style=filled, fillcolor=green]
    n00000019:port1 -> n0000001c [style=dashed]
    n0000001c [label="Intel IPU6 BE SOC capture 0\n/dev/video0", shape=box, style=filled, fillcolor=yellow]
}