search

linux-surface / linux-surface

Linux Kernel for Surface Devices
4.5k stars 198 forks source link

Camera support #91

Closed jrevillard closed 3 months ago

jrevillard commented 4 years ago

I see that there is some activity here: https://github.com/jakeday/linux-surface/issues/145

I think it's better to trace it here isn't it ?

Best, Jerome

Maintainer note: There is a BountySource bounty available for this issue. If you'd like to see a camera driver consider donating to incentivise the work.

mmalmeida commented 4 years ago

I agree

qzed commented 4 years ago

Right, makes sense to track this here.

archseer commented 4 years ago

Copying my old comment here:

I found libcamera based on this bug report, it seems to have the required userspace code to have ipu3 working on 5.0. libcamera FOSDEM slides

archseer commented 4 years ago

Unlikely to get anywhere without someone stepping up and putting some work into it though.

I was hoping to work on the cameras before buying my Surface Laptop 3, but since the newest generation's cameras are available via USB it just worked out of the box -- I don't have hardware to develop against.

I think @GrayHatter was taking a look at things a while ago.

koroki commented 4 years ago

Also Surface go has not support for camera.

It helps to me a lot because I use Surface Go basically for working outside the office and, sometime I have to do some Skype videocall... And I should do in my mobile.

Doridian commented 4 years ago

Well, I managed to find datasheets for the two missing drivers at least. Never done kernel dev, but I mean, how hard could it be? (sarcasm). Eh, due to obvious current events I have a bit more free time than usual, so let's see where I can get... Put random stuff in my repo here: https://github.com/Doridian/sb2-linux-cameras Will probbaly put random C files in there if I managed to get anywhere at all.

qzed commented 4 years ago

I wish you the best of luck! For reference, here are the (I think) most relevant links from the other discussion:

Getting started with the ovXXXX drivers is probably the best idea.

archseer commented 4 years ago

@Doridian Feel free to message us on IRC (freenode @ ##linux-surface) or the Matrix bridge (might be easier to join), we can always help out!

I also recommend starting with https://github.com/linux-surface/linux-surface/issues/91#issuecomment-588275735 and doing the OV drivers first. Once we have those we can figure out how to integrate with ipu3/libcamera. I was searching for comparable drivers by looking at the product brief's functional diagram. I see that you also found the datasheet for both which is pretty great.

GrayHatter commented 4 years ago

@Doridian highlight me with nearly anything you think is relevant, I'm currently working on getting it working, but haven't had the spare cycles lately.

That's awesome to find the datasheets for the cameras, I'm pouring through them now.

Currently, my understanding, and thus next step, is to figure out what magic windows is using to load the cameras after boot, the DSDT is incomplete on the SB2. I originally thought you'd have to load a custom DSDT at boot up, but https://www.kernel.org/doc/html/v5.2/firmware-guide/acpi/method-customizing.html seems to imply we can do what windows does and inject our fixes at runtime.

Then IIRC we write an i2c driver for the cameras to control them, and then wire them up to ipu3, like @archseer said

It's all been done before, so it shouldn't be too much actual work, once we find the right magic to apply.

Doridian commented 4 years ago

@GrayHatter Good ideas. I'm currently just poking around the Windows driver package. There seems to be quite a bit of hardcoding some things going on in there... I think the MSHW01*(0,1,2) are power related, because they are close to CAMP in the DSDT but... I still haven't found how they map the ipu3 to the cameras, but I assume all the same silliness... Kwn5ffu

qzed commented 4 years ago

Based on the DSDT it looks like MSHW014x/MSHW015x are the subsystem IDs (_SUB) for the CAMF/CAMR/CAM3 devices (x=0 is CAMF, x=1 is CAMR, x=3 is CAM3). They don't look like own devices as those names are provided as return values to the GCSM function which is only called from _SUB and _DSM. The CAMx seem to be the actual I2C sensor controllers, as they have a _CRS with an I2cSerialBusV2 resource. Thus the function you've posted seems to check if the camera controller is supported (or at least if it's any "valid" combination of sensor and subsystem ID). I'm not quite sure why they don't just use the OVxxxx ID (or the _HID field) but additionally rely on the other IDs.

Doridian commented 4 years ago

@qzed Ah, makes sense. Is there and HWIDs for the ipu3 interface or other things I could try searching for in the driver to see how it fetches those devices? This is the first DSDT I have ever looked at, and I am kind of learning as I go here.

qzed commented 4 years ago

I haven't really figured out much more than that. There are also the INT3472 devices that I mentioned in the other thread, but they seem to be controllers for the camera flashes/LEDs. I have no idea how the sensors are matched up to the ipu3 interface. The ipu3 itself is a PCIe device, but I haven't found anything for it specifically in the DSDT.

theGeekyLad commented 4 years ago

What's the approach like to get the camera working on the Surface Go? I ain't no pro but is there some kind of support that I can provide? Also the readme mentions partial support - any idea how it's enabled?

spoorun commented 4 years ago

Happy to fund this if someone wants to get it resolved speedily... Bountysource? Or what's best? £60//$75/€70

qzed commented 4 years ago

The process on the Go isn't any different than for the other devices, except that it uses a different sensor (OV2740).

archseer commented 4 years ago

@olantrust IssueHunt or Bountysource are all fine. IssueHunt seems to do some reward matching on their end as well to increase the bounty.

fabwu commented 4 years ago

I've copied the ov5675.c and replaced the ACPI id on line 1226 with INT33BE.

Now I see the following error message

ov5693 i2c-INT33BE:00: can't get clock frequency
ov5693 i2c-INT33BE:00: failed to check HW configuration: -22
ov5693: probe of i2c-INT33BE:00 failed with error -22

but at least the probe function gets called ^^

spoorun commented 4 years ago

@olantrust IssueHunt or Bountysource are all fine. IssueHunt seems to do some reward matching on their end as well to increase the bounty.

Great. If you or one of the crew can add the repo to the Issue Hunter app, then we'd happily support what we can financially with this and maybe other issues...

spoorun commented 4 years ago

$75 Dollar bounty provided:

https://www.bountysource.com/issues/88806607-camera-support

mmalmeida commented 4 years ago

$75 Dollar bounty provided:

https://www.bountysource.com/issues/88806607-camera-support

Contributed!

qzed commented 4 years ago

@fabwu: I think you're on the right track. I don't have any specific insight to the ovXXXX drivers, but I think the way forward is looking at the in-kernel driver and adapting for the spec differences. The ov5675 one fails because it can't read the configuration from ACPI. Linux expects this in the _DSD (https://uefi.org/sites/default/files/resources/web-page-v2.pdf) method of the device. Maybe you can get the DSDT/_DSD of a device where the sensor works and (for now) hard-code those values (you'll probably have to adapt them for your model). Maybe it's also possible to dig out those values from Windows or the driver packages.

fabwu commented 4 years ago

@qzed Thanks for your answer I'm a bit less confused now. As the _DSD is required by ov5675.c I don't think I should go in that direction. Do you know an easy way to verify if the device is even powered on?

I tried to read some values via I2C but I've just got errors. There is a CAMP in the dsdt can I call one of its methods?

qzed commented 4 years ago

You can manually call all ACPI functions from kernel code via the acpi_evaluate_object or other functions using that (e.g. acpi_evaluate_dsm_typed for _DSMs). The handle parameter can be NULL if the method parameter is a fully qualified path starting with \ (otherwise it's local).

In the case of a PowerRegion, you can actually check _STA directly from sysfs, via e.g. cat /sys/bus/acpi/devices/INT33BE:00/power_resources_D0/LNXPOWER:0c/status. Honestly, I wouldn't worry too much about that stuff, as that usually works (this is basically generic ACPI/device stuff).

I think reading from the I2C chip should work pretty much out of the box via a simple I2C driver. If you run into any errors when reading via a kernel driver, feel free to contact me on IRC (I just think some progress infos are better kept here on github for the record, but IRC is probably better for that so that we don't spam this issue). The chip-ID registers SC_CHIP_ID, SC_SCCB_ID, SC_CHIP_REVISION are probably a good start. You should be able to just pretty much copy this function call and change 24BIT to 16BIT to get the first one (the register address for that one seems to match, you should verify the other addresses via the doc though). I don't think any specific setup is required to get the I2C device running, as far as I can see the ACPI subsystem should take care of that based on the definition (see i2c-core-acpi.c).

kachapman commented 4 years ago

$75 Dollar bounty provided: https://www.bountysource.com/issues/88806607-camera-support

Contributed!

Contributed!

anidel commented 4 years ago

$75 Dollar bounty provided: https://www.bountysource.com/issues/88806607-camera-support

Contributed!

Contributed!

Same. Thank you all!

spoorun commented 4 years ago

Bounty now at $145 :D

mojth commented 4 years ago

I've never done kernel development, so I can't help you with that, but I also contributed $15. If there is any thing else I can do to help, let me know.

qzed commented 4 years ago

@fabwu Seems you were right, the sensor needs to be configured/powered on/enabled or something. There are some GPIOs (https://github.com/linux-surface/acpidumps/blob/master/surface_book_2/dsdt.dsl#L16108-L16125) which need to be set for this to work. Specifically, the first two have to be on (1), then reading the register via basic I2C commands (no need to use SCCB stuff) works. The third GPIO is the activity indicator, setting it to 1 tuns on the indicator led.

The device they belong to seems to be a TPS68470 (https://www.ti.com/lit/ds/symlink/tps68470.pdf?ts=1588361938977), which is a power management unit for compact camera modules.

It looks like Intel has already done some work on that in the past (https://github.com/intel/intel-camera-drivers/blob/e6392058d52c0d2777741f9d4dbe7c4de7a3951a/drivers/media/platform/intel-ipu4/intel-ipu4-acpi.c). Based on that, the first GPIO pin is XSHUTDN of the OV sensor (search for xshutdown in the driver). So it doesn't seem to be the exact device presented in the data sheet, as this has no xshutdown pin. Looks like there's also a bit more info about the ACPI fields in there (e.g. SSDB). Also there seem to be more I2C devices encoded in the _DSM (see intel_ipu4_get_acpi_devices and get_i2c_info).

alex-a-soto commented 4 years ago

$55 Dollar bounty provided, Bounty now at $200.

https://www.bountysource.com/issues/88806607-camera-support

spoorun commented 3 years ago

Bounty now at $270...

Thanks @alex-a-soto, @anidel @kachapman @mmalmeida etc... :D

spoorun commented 3 years ago

Bounty @ $320 now, hopefully that will inspire and help us have video access for lockdown work at this time... :D

kachapman commented 3 years ago

Bounty @ $320 now, hopefully that will inspire and help us have video access for lockdown work at this time... :D

I had to pickup another webcam to use in the interim!

danieldeng2 commented 3 years ago

Is it true now SL3 has camera support? Any chance to replicate on SP7?

archseer commented 3 years ago

SL3 has a camera that seems to be wired via USB internally. As such it worked out of the box on the stock kernel with no patching. SP7 uses the same camera design the previous models used which requires all the work outlined in this thread.

Dzeri96 commented 3 years ago

This might be slightly off-topic, I've been checking up on this issue and even wanted to donate, but the bountysource website seems to be down 95% of the time. Anyone having the same issue? It seems pretty unreliable

concreted commented 3 years ago

@Dzeri96 Been having the same issue.

twinkybot commented 3 years ago

Bounty now at 360$ I hope there is someone willing to fix this properly. Fucking MS and their propriety shit. MS loves FLOSS, my ass!

jhand2 commented 3 years ago

@qzed @fabwu @Doridian chiming in here to offer my help. I know basically nothing about camera drivers but I've been combing through some of the details you've all posted here, as well as the existing ov* drivers and I'm happy to see if I can make any progress.

My plan is to start with the ov5693 driver.

You've mentioned starting with an existing driver (ov5675 and ov5695) and modifying as necessary. Is there something in particular in the camera specs or driver code you're looking for to figure out which ov driver should be somewhat similar to the ov5693? If not, my plan is to start trying to understand the ov5675 as @fabwu has done and go from there.

Since I'll presumably be stuck working from home for a long time, getting the camera working would be very nice :smile:

EDIT: oh I understand the previous comments now. ov5675 tries to read clock frequency (and other values) from the _DSD, but there is no _DSD for the ov5693 entry in the DSDT table. So maybe trying to find an OV driver that doesn't depend on _DSD might make more sense. Or just hard-coding the values retrieved from some TBD location.

fabwu commented 3 years ago

@jhand2 Sorry for keeping you waiting so long but I accidentally deleted my testing code and I had to redo it form scratch.

It's great to hear that you are willing to help. I prepared a repository with my progress here:

https://github.com/fabwu/ov5693

I copied most of the code from the Intel driver qzed mentioned. The only thing my code does is reading the available information from ACPI. Finally, these lines turn the status LED on.

The next step is to get the i2c communication working. qzed has already done that by switching on the first two GPIOs (xshutdown and custom_gpio[0] in my code).

It would be great if you could setup i2c and read out e.g. the chip id. For a start, you could copy parts from the ov5695 driver and adapt them for the ov5693.

The next challenge is then to implement the Video4Linux API but I have no clue how that works...

jhand2 commented 3 years ago

@fabwu looks like we have done some similar work :smile: https://github.com/jhand2/surface-camera

I used a combination of the ov5670 driver and the ov5693 atomisp driver as a template for this one. I understand some of the v4l2 stuff, but am still learning more.

I am currently working on modifying the DSDT to allow the ipu3-cio2 driver to discover the sensor v4l2 subdev. You can see the changes I've made to dsdt.dsl (search for _DSD to see where I made changes), although currently the cio2 driver does not associate the PCI device with the CIO2 dsdt node, so that's not working right now. Hopefully in the next couple days I can figure out why it isn't working.

This doc helped my understanding of how v4l2 subdevices fit into the equation quite a bit: https://www.kernel.org/doc/html/v4.10/media/kapi/v4l2-subdev.html

To answer the above, I was able to power on the camera and read the chip id over i2c. Also I got the led to turn on and off which was quite satisfying :smile:

archseer commented 3 years ago

Great work!

From what I'm understanding, the data should be raw sensor data that needs to be processed by ipu3. I'm not sure what the exposed interface should be, but you should probably be integrating against libcamera

How is libcamera different from V4L2? We see libcamera as a continuation of V4L2. One that can more easily handle the recent advances in hardware design. As embedded cameras have developed, all of the complexity has been pushed on to the developers. With libcamera, all of that complexity is simplified and a single model is presented to application developers.

archseer commented 3 years ago

Ah okay, looks like libcamera still uses parts of the v4l2 subdevice layer:

The kernel drivers control the camera hardware and expose a low-level interface to userspace through the Linux kernel V4L2 family of APIs (Media Controller API, V4L2 Video Device API and V4L2 Subdev API).

fph commented 3 years ago

FYI, the Linux media maintainer submitted a pull request for the kernel last week https://lkml.org/lkml/2020/6/3/179

  • The atomisp staging driver was resurrected. It is meant to work with 4 generations of cameras on Atom-based laptops, tablets and cell phones. So, it seems worth investing time to cleanup this driver and making it in good shape.
jhand2 commented 3 years ago

It is meant to work with 4 generations of cameras on Atom-based laptops

SB2 and other surface machines are not Atom-based laptops, they instead use the Intel "Core" series CPUs. So unfortunately the atomisp stuff doesn't help us.

That said, I have made some good progress and feel I am close to getting the front-facing camera working. I should have something to share in the next week or two.

kitakar5525 commented 3 years ago

FYI, the Linux media maintainer submitted a pull request for the kernel last week https://lkml.org/lkml/2020/6/3/179

  • The atomisp staging driver was resurrected. It is meant to work with 4 generations of cameras on Atom-based laptops, tablets and cell phones. So, it seems worth investing time to cleanup this driver and making it in good shape.

@qzed Surface 3 (Intel Atom Cherry Trail) doesn't use ipu3 but does use atomisp. And sensors used on S3 are different than other "Core" series Surfaces (Namely, AR0330 (ACPI ID: APTA0330) and OV8835 (ACPI ID: OVTI8835)). Should I track Surface 3 camera support in this issue or should I open a new issue?

kitakar5525 commented 3 years ago

Leaving here what I found in firmware download file for SB1.

Intel(R) Imaging Signal Processor 2500

Intel(R) CSI2 Host Controller

Intel(R) AVStream Camera 2500

Intel(R) Control Logic

Microsoft IR Camera Front

Microsoft Camera Front

Microsoft Camera Rear

koroki commented 3 years ago

FYI, the Linux media maintainer submitted a pull request for the kernel last week https://lkml.org/lkml/2020/6/3/179

  • The atomisp staging driver was resurrected. It is meant to work with 4 generations of cameras on Atom-based laptops, tablets and cell phones. So, it seems worth investing time to cleanup this driver and making it in good shape.

good news!

qzed commented 3 years ago

@kitakar5525 I think the atomisp should be tracked separately. I suspect this is quite a bit different.

kitakar5525 commented 3 years ago

@kitakar5525 I think the atomisp should be tracked separately. I suspect this is quite a bit different.

OK. I've opened a separate issue for S3 cameras: https://github.com/linux-surface/linux-surface/issues/209