search

jakeday / linux-surface

Linux Kernel for Surface Devices
2.59k stars 241 forks source link

Surface Pro 4 / 2017 and Books: Cameras are not working #145

Open L4SCHI opened 6 years ago

L4SCHI commented 6 years ago

Hey! Thank you very much for your work. I've installed the kernel version 4.15.17-surface-linux-surface on Ubuntu 17.10. I am using the MS Surface Pro 4 with 8GB RAM and 256 GB Storage. All devices work very fine. Just the camera is not working. Is it possible to solve this with patch? Can anybody help me?

Thank you

jakeday commented 6 years ago

@L4SCHI I haven't finished the patches for the camera on the Surface Pro 4 and newer yet. I'll work on it after finishing the i2c driver for acpi.

pysongh commented 6 years ago

Thank you for all your work. I installed latest version (4.17.3) but, webcam doesn't work. Is it fixed?

jakeday commented 6 years ago

@pysongh No, not yet.

Reichenbachian commented 6 years ago

Sorry to be a bother, but the description says that it has partial support, it's just 'disabled for now'. How may one go about re-enabling it, so as to attempt to help fix it?

vnenkpet commented 6 years ago

Are there any news on this? This would be greatly appreciated! Currently an annoyance to the point I'm wanting to buy a non-Surface device unfortunately... I don't have any experience with Linux kernels, but still, any way we could help?

denzilferreira commented 6 years ago

I found this today. I'm compiling the kernel from sources as I'm on Fedora 28 and I'll report back if this works to bring the cameras online:

https://github.com/shvr/fedora-surface-pro-3-kernel/blob/master/Microsoft-Surface-Pro-4-Surface-Book-camera-support.patch

Seems to be a relatively simple patch, adding the vendor IDs to the driver.

denzilferreira commented 6 years ago

Seems to be more than matching IDs :) @jakeday has already these identifiers on his camera patch file.

I found some links that may help Jake to get this up and running: Driver for front camera (OV5693 - INT33BE): https://elixir.bootlin.com/linux/v4.14.71/source/drivers/staging/media/atomisp/i2c/ov5693 Driver for rear camera (OV8865 - INT347A): https://github.com/lenovo-yt2-dev/android_kernel_lenovo_baytrail/blob/357b3bc165c76b9cf1f0d2c08e458576018164a3/drivers/external_drivers/camera/drivers/media/i2c/ov8865.c Driver for IR camera (OV7251 - INT347E): https://github.com/ADVANSEE/0066_linux/blob/ba2479578aa7f35be22f6749f7504ba3a68414dc/drivers/media/video/mxc/capture/ov7251_mipi.c

grokkingStuff commented 6 years ago

@jakeday, hey there, love the work you're doing!

@denzilferreira, any idea how to use the files?

holger-dev commented 5 years ago

After getting the battery-reading work, let's do the next step for the cameras :). The last post from @denzilferreira looks interesting, but i'am not able to program in c. Someone else?

qzed commented 5 years ago

@htrampe I might have a look at that, now that I think I've got the ethernet on the dock working. WIFI is also still a major issue I haven't looked at, but IIRC Jake said he's disabled power-save in the driver which could fix that, so I guess I'll look at the cameras next.

kitakar5525 commented 5 years ago

ov5693 and atomisp (, which is needed for ov5693 for now) modules once existed in staging tree until 4.17, removed in 4.18 due to a lack of progress. I happened to use linux-lts-4.14.91 with cameras.patch, but ov5693 is not working.

$ dmesg | grep ov5693
[    3.071388] ov5693: module is from the staging directory, the quality is unknown, you have been warned.
[    3.291951] ov5693 i2c-INT33BE:00: gmin: initializing atomisp module subdev data.PMIC ID 1
[    3.295343] ov5693 i2c-INT33BE:00: gpio failed 2
[    3.302353] ov5693 i2c-INT33BE:00: sensor power-up failed
[    3.305993] ov5693 i2c-INT33BE:00: gpio failed 2
[    3.309088] ov5693 i2c-INT33BE:00: sensor power-up failed
[    3.312020] ov5693 i2c-INT33BE:00: gpio failed 2
[    3.314761] ov5693 i2c-INT33BE:00: sensor power-up failed
[    3.317552] ov5693 i2c-INT33BE:00: gpio failed 2
[    3.320905] ov5693 i2c-INT33BE:00: sensor power-up failed
[    3.323576] ov5693 i2c-INT33BE:00: gpio failed 2
[    3.326231] ov5693 i2c-INT33BE:00: ov5693 power-up err.
[    3.329417] ov5693 i2c-INT33BE:00: gpio failed 2
[    3.332436] ov5693 i2c-INT33BE:00: sensor power-gating failed
[    3.335515] ov5693: probe of i2c-INT33BE:00 failed with error -22

Just for your information.

qzed commented 5 years ago

Yeah, unfortunately it won't be that easy. I've had a look at this over the past couple days and I still don't quite understand things as good as I'd like to get started.

Here's a rough overview: OV5693 is a camera sensor with an I2C interface. This is in fact the actual camera sensor, but it doesn't directly provide the images. The I2C interface is for stuff like setting the exposure etc., the actual images are provided via a MIPI CSI-2 (Camera Serial Interface) connection handled by a PCI device (with ID 8086:9d32). This device is shared by all cameras (on KabyLake it can handle 4 of them parallel, on SkyLake 2 or 3, AFAIK depending on the configuration).

Then there are the INT3472 devices, which are defined in the ACPI directly next to each camera sensor. According to the kernel, those are TPS68470, which is a "Power Management IC (PMIC) with flash LED driver and ref clock generation for compact camera module", so that would fit in.

Also there's the matter of the ISP (Image Signal Processor). This is also a PCI device (8086:1919), and if I understand that correctly, independent of the CSI-2 interface.

There's a bit about all of that here (Section 3.4), but not really much to go on.

As to drivers: There's currently some work upstream for the ISP device, it's currently in staging for 4.21. I'm not yet sure if we actually need that, or if we can get the images directly from the CSI-2 interface. The CSI-2 interface has an upstream driver, but I'm not sure if that's working properly or supports everything. I've enabled it in the kernel options but nothing changes with respect to video devices under /dev. The TPS68470 also has a driver upstream, but the same problem there. In this case, there are 3 GPIO ouput lines defined in ACPI, but the driver expects an additional 4 input lines. Maybe it's working, maybe not, haven't had the time to check that yet.

Then there's the OV5693 driver, or rather the one that was in staging. This depends on the atomisp subsystem and won't work with anything else without modification. The atomisp is a different signal processor for some older Intel Atom platform, but we might be able to use that as a template, AFAIK this also uses some CSI-2 PCI interface. The catch is: This has apparently been removed because the atomisp subsystem was a bit of a mess and Intel didn't care enough to fix it.

So with quite a bit of luck we only need to wait some time and adapt the controller driver for OV5693 to work without the atomisp subsystem.

qzed commented 5 years ago

Also the CIS-2 interface and ISP combined are sometimes also referred to as IPU (Image Processing Unit), which doesn't really further understanding when it comes to what handles what. Here's a somewhat similar device setup, which can help us figure this out.

andreivolt commented 5 years ago

I'm also interested in camera support - and props to @qzed for the research you've done. You're saying it "won't be that simple", since this is OSS I suspect someone will tackle it at some point (someone with kernel experience so not me :)), what are the odds you'd put on availability say 3 to 6 months for now?

Basically I'd like to know if there's hope or just to forget about it, seeing it's also not seen as vital to many people. It's just a bit frustrating to know it's there and good but unavailable...

qzed commented 5 years ago

@andreivolt Sorry it took me so long to answer, unfortunately I'm a bit busy at the moment.

I hope that the big parts will be supported by Intel (i.e. IPU and CIS2-interface, as mentioned there seems to be some development at the moment), and I think we should then be able to fill in the missing parts. I'd really like to have working cameras myself, but I can't give you any credible estimate. I just still know too little about that, but I'll have a look at it again once I've got a bit more time to spare.

I think if the support on the Intel side is there, it could be possible that we have camera support in 6 months, but don't hold me to that.

andreivolt commented 5 years ago

@qzed no problem and thank you for that estimate, it seems you've got a skeleton of a solution, it seems improbable that it won't happen at some point, which is great news after reading elsewhere it's not likely to ever happen

Thanks again for all your work here!

woopstar commented 5 years ago

Would donations in some way make this bump through faster?

qzed commented 5 years ago

For me: No. Sorry. I'm currently focusing on a proper dGPU power-on/power-off switch (as for me this is the faster-to-fix and more pressing issue right now) and some SB1/SP4 stuff on the side. At the moment I can't even tell you with full confidence what's missing and what's already implemented.

I'll probably try to have another look at it after the dGPU stuff is done, but if someone else wants to take this on, be my guest.

qzed commented 5 years ago

I haven't looked at it in-detail yet, but there is now some documentation for the ipu3 driver currently being worked on by Intel: https://www.kernel.org/doc/html/latest/media/v4l-drivers/ipu3.html. Looks very promising so far.

Also here's some discussion about the ipu3 camera subsystem: https://www.mail-archive.com/linux-media@vger.kernel.org/msg133794.html.

floion commented 5 years ago

Hi all, I'm using the acpi and camera patches from 4.18.16-1 on a 4.18.14 kernel I had to re-add the atomisp driver as it's been removed in 4.18 However, with the acpi and camera patches I don't seem to get any /dev/i2c* devices created so atomisp_ov5693 can't do it's job. Am I missing something for i2c?

Thanks for this project

qzed commented 5 years ago

However, with the acpi and camera patches I don't seem to get any /dev/i2c* devices created so atomisp_ov5693 can't do it's job.

The actual i2c devices can be found under /sys/bus/i2c/devices/, so the devices are all present.

The problem at the moment is that the drivers that were in staging are (as far as I remember) dependent on the atomisp platform, and that there is some work required in getting the ipu3 driver (the one actually responsible for receiving and processing the image) and the drivers for the sensor controllers to play together (see mailing list links above). The camera stuff depends on multiple devices and drivers to work together, so the issue is a bit more complex than just a single missing driver.

Unfortunately, I haven't had much time lately, so I haven't managed to look into it as much as I wanted.

floion commented 5 years ago

Hi @qzed in my case the ov5693 driver does not start as https://github.com/jakeday/linux-surface/issues/145#issuecomment-451140904 pasted. It only displays

atomisp_ov5693: module is from the staging directory, the quality is unknown, you have been warned.

There's no further probing of the i2c drivers like above, just driver starting and that's it

qzed commented 5 years ago

@floion Quoting from the module description, "It currently only works with the atomisp driver.". Atomisp is a ISP (Image Signal Processor) for the Intel Atom platform. This driver as included here wont work without modifications on the Surface devices (never has), doesn't really matter where it fails, it will fail simply due to the different hardware. Maybe @kitakar5525 had some other atomisp related stuff enabled that allowed the driver to probe, maybe it's a model-specific thing that allowed it to load, or maybe there have been some other related changes upstream since then.

Even if that driver would work, the binding to the ipu3 driver is missing.

floion commented 5 years ago

The OV5693 driver included in the camera.patch won't even compile because it includes v4l2-chip-ident.h which is an old header only present in kernel 3.x series. So what I did for my kernel (4.18.14) was to revert the removal of atomisp (which also included an OV5693 driver). It is this atomisp-ov5693 for which I pasted above. One thing that I had to enable in my kernel was CONFIG_MFD_INTEL_LPSS_PCI. Had to do some look-up to get to this in lack of schematics for the Surface Pro. And with this lpss driver enabled I now get

i2c-INT33BE:00 -> ../../../devices/pci0000:00/0000:00:15.2/i2c_designware.2/i2c-2/i2c-INT33BE:00

in /sys/bus/i2c/devices. However, now some driver trouble:

[ 2.512117] BUG: unable to handle kernel NULL pointer dereference at 0000000000000058 [ 2.512118] PGD 0 P4D 0 [ 2.512121] Oops: 0002 [#1] PREEMPT SMP PTI [ 2.512123] CPU: 5 PID: 562 Comm: systemd-udevd Tainted: G C 4.18.14-yocto-standard #1 [ 2.512124] Hardware name: Microsoft Corporation Surface Pro 6/Surface Pro 6, BIOS 234.2344.769 09/08/2018 [ 2.512128] RIP: 0010:gmin_camera_platform_data+0x30/0x60

and the kernel oops'es. To be noted, I also applied https://github.com/jakeday/linux-surface/pull/208/commits/690403acbcd633517432915604ec651af28baf48

Without applying the above, the atomisp-ov5693 driver would complain that it can't get INT33BE:00_CamClk, INT33BE:00_ClkSrc, INT33BE:00_I2CAddr: https://github.com/torvalds/linux/blob/v4.17/drivers/staging/media/atomisp/platform/intel-mid/atomisp_gmin_platform.c#L721

So some progress here

qzed commented 5 years ago

@floion I'm not sure what you're trying to achieve here. If you want to adapt the ov5693 driver for the Surface/general use, I'd suggest you have a look at the other ov5xxx drivers, for example the one for ov5695.

The problem you're running into right now is that the atomisp-ov5693 is designed for an Intel Atom SOC based platform (and only for that, stuff like atom-based netbooks and tablets).

AFAIK Jake has been trying to work around that (which is why the patches change the error). In my opinion, though, the right way forward is to implement the ov5693 driver independent of the platform (similar to all the other media-i2c/ov drivers). I really don't think trying to modify the platform driver to work on a completely different platform is a good idea (and also not likely to be easy or work at all) and I highly doubt that such modifications will ever be accepted upstream.

As I've already mentioned, to ultimately get the cameras working, we need proper (read platform-independent) drivers (similar to the ones already upstream) for the sensor controllers (ov5693, ov8865, ov7251) and a bridge to the ipu3 driver.

kitakar5525 commented 5 years ago

It seems that, at least, ov7251 is already upstream. I built the module with this config:

# CONFIG_MEDIA_SUBDRV_AUTOSELECT is not set
CONFIG_VIDEO_OV7251=m

However, not probed at all. No logs in dmesg even if I load the module manually (not loaded by default) by sudo modprobe ov7251🤔 (EDIT: tested on Linux 5.2 without 0004-cameras.patch)

kitakar5525 commented 5 years ago

I found a bug report of the ov7251 on Kernel.org Bugzilla.

archseer commented 5 years ago

Dumping some more notes, I currently do not own a Surface device, but I'm vetting it for potential purchase.

I found libcamera based on this bug report, it seems to have the required userspace code to have ipu3 working on 5.0. libcamera FOSDEM slides

kitakar5525 commented 4 years ago

Does anyone know why the upstream driver ov7251 does not probe the device at all as mentioned before? and libcamera does not enumerate any cameras:

$ sudo cam -l
[21:13:08.805213608] INFO Camera camera_manager.cpp:94 libcamera v0.0.0+960-744fabcb
Available cameras:

What's the difference between of_device_id (ov7251) and acpi_device_id (ov5670 or ov8856) device matching regarding device probe?

I modified the matching to use acpi_device_id instead:

diff

```diff diff --git a/drivers/media/i2c/ov7251.c b/drivers/media/i2c/ov7251.c index 0c10203f8..1ec27bf87 100644 --- a/drivers/media/i2c/ov7251.c +++ b/drivers/media/i2c/ov7251.c @@ -1481,15 +1481,18 @@ static int ov7251_remove(struct i2c_client *client) return 0; } -static const struct of_device_id ov7251_of_match[] = { - { .compatible = "ovti,ov7251" }, +#ifdef CONFIG_ACPI +static const struct acpi_device_id ov7251_acpi_ids[] = { + {"INT347E"}, { /* sentinel */ } }; -MODULE_DEVICE_TABLE(of, ov7251_of_match); + +MODULE_DEVICE_TABLE(acpi, ov7251_acpi_ids); +#endif static struct i2c_driver ov7251_i2c_driver = { .driver = { - .of_match_table = ov7251_of_match, + .acpi_match_table = ACPI_PTR(ov7251_acpi_ids), .name = "ov7251", }, .probe_new = ov7251_probe, ```

but cannot initialize the device:

kern  :err   : [76976.135409] ov7251 i2c-INT347E:00: endpoint node not found
kern  :warn  : [76976.137010] ov7251: probe of i2c-INT347E:00 failed with error -22

Maybe not intended to be used with ipu3?

baconlt commented 4 years ago

I am just checking into this discussion - just acquired a SP4 and loving all that works under Ubuntu. However has there been any progress on the front-facing camera? My son will need it for his Zoom sessions if we go full virtual and I may just have to buy a separate webcam.

archseer commented 4 years ago

This repo is unmaintained, please follow https://github.com/linux-surface/linux-surface/issues/91 for current discussion and progress. There was some promising developments recently but no driver is available yet.

nlegrand92 commented 4 years ago

How can I get my volume control to work? I can't use my buttons for volume

thatonearchguy commented 4 years ago

What device are you using, I'm on a surface pro 6 using the linux-surface/linux-surface kernel and it's working perfectly for me

tellhandel commented 3 years ago

On the Surface Book I've noticed a change since the first versions of kernel-5.9.xx-surface during the loading process (a.i. wasn't there with the 5.3.18). The light near the camera flashes once (see pic). Is it due to the "intel_backlight.service" command during the loading process? False hope that a solution is approaching?

kernel-surface-system-backlight

archseer commented 3 years ago

Wrong repository, if you're running 5.1+ -surface kernels you're running linux-surface.

There's been a lot of work done on camera drivers: https://github.com/linux-surface/linux-surface/issues/91

5.10 has initial support for certain camera models, but further work is needed to get a picture with proper white balance: https://github.com/linux-surface/linux-surface/issues/96#issuecomment-746592521

gaurava05 commented 2 years ago

@jakeday Hey, thanks for your work. Is the camera support for Surface Book 3 down the roadmap anytime soon?

kbingham commented 2 years ago

Surface cameras are supported/documented by the linux-surface project at

https://github.com/linux-surface/linux-surface/wiki/Camera-Support

SB3 is documented as using the IPU4. This can not currently be supported without extensive kernel driver development, requiring either direct support from Intel or ... a lot of funding / developer time.

gaurava05 commented 2 years ago

Thanks @kbingham , yeah makes sense.