Camera support

[jrevillard]

I see that there is some activity here: https://github.com/jakeday/linux-surface/issues/145

I think it's better to trace it here isn't it ?

Best, Jerome

---

Maintainer note: There is a BountySource bounty available for this issue. If you'd like to see a camera driver consider donating to incentivise the work.

[kitakar5525]

I saw your github repo. It seems that you didn't bump OEM revision. This is the reason the new DSDT isn't working.

Try this:

diff --git a/dsdt-mods/lenovo_miix_510/dsdt.dsl b/dsdt-mods/lenovo_miix_510/dsdt.dsl
index af399e5..2aa18e6 100644
--- a/dsdt-mods/lenovo_miix_510/dsdt.dsl
+++ b/dsdt-mods/lenovo_miix_510/dsdt.dsl
@@ -18,7 +18,7 @@
  *     Compiler ID      "ACPI"
  *     Compiler Version 0x00040000 (262144)
  */
-DefinitionBlock ("", "DSDT", 2, "LENOVO", "CB-01   ", 0x00000001)
+DefinitionBlock ("", "DSDT", 2, "LENOVO", "CB-01   ", 0x00000002)
 {
     /*
      * iASL Warning: There were 17 external control methods found during

[djrscally]

I did actually notice that I'd forgotten to do that, but my original mod to the DSDT table worked anyway - I'd added an I2cSerialBusV2 to my INT3472 device entry, and that worked fine without the version increase. Not sure why. Anyway, I applied the patch and tried again, still no dice. I pushed the modifications that I made to my repo now too. Just recompiling with those other patches from @jhand2.

EDIT: Hmm, the thing I did copy directly from yours was the UUIDs, as I'm under the impression they just have to be random and not conflict with existing ones, so it seemed like that would be ok. Maybe that's wrong.

Thanks for the help, by the way :D

[kitakar5525]

How have you resolved these errors in DSDT? I guess the compilation didn't pass actually.

$ iasl -tc dsdt.dsl 2>&1 | grep -ie "error" -n2
dsdt.dsl  38487:                 Acquire (^^WMI1.MWMI, 0xFFFF)
Error    6163 -                                    ^ Object is created temporarily in another method and cannot be accessed (^^WMI1.MWMI)

dsdt.dsl  38496:                 Release (^^WMI1.MWMI)
Error    6163 -                                    ^ Object is created temporarily in another method and cannot be accessed (^^WMI1.MWMI)

For now I added this commit: https://github.com/kitakar5525/surface-ipu3-cameras/commit/898cdff9f54aa3bae62ab86d97c2a8609f893fc4 at least compilation now works.

[djrscally]

I don't even get those errors!

ASL Input:     dsdt.dsl - 38690 lines, 1225152 bytes, 18504 keywords
AML Output:    dsdt.aml - 154853 bytes, 3370 named objects, 15134 executable opcodes
Hex Dump:      dsdt.hex - 1452108 bytes

Compilation complete. 0 Errors, 478 Warnings, 109 Remarks, 374 Optimizations

Version differences perhaps? That's something called out on the Arch wiki page. My compiler version info:

djrscally@djrscally-MIIX-510-12ISK:~/Coding/miix-510-cameras$ iasl -v

Intel ACPI Component Architecture
ASL+ Optimizing Compiler/Disassembler version 20180105
Copyright (c) 2000 - 2018 Intel Corporation

[kitakar5525]

Maybe mine is too new:

$ iasl -v

Intel ACPI Component Architecture
ASL+ Optimizing Compiler/Disassembler version 20200717
Copyright (c) 2000 - 2020 Intel Corporation

EDIT: Hmm, the thing I did copy directly from yours was the UUIDs, as I'm under the impression they just have to be random and not conflict with existing ones, so it seemed like that would be ok. Maybe that's wrong.

Yes, it's ok. UUIDs in DSDT are the same for all the devices.

[djrscally]

Progress! This is with the other patches from @jhand2 's repo applied:

djrscally@djrscally-MIIX-510-12ISK:~/Coding/miix-510-cameras$ media-ctl -d /dev/media0 -e "ov2680 7-0010"
/dev/v4l-subdev6
djrscally@djrscally-MIIX-510-12ISK:~/Coding/miix-510-cameras$ cam -l
[0:03:18.745748269] [3643]  INFO Camera camera_manager.cpp:287 libcamera v0.0.0+1735-54ca4696
[0:03:18.773773228] [3644]  INFO IPU3 ipu3.cpp:813 Registered Camera[0] "\_SB_.PCI0.CAM1" connected to CSI-2 receiver 1
Available cameras:
1: \_SB_.PCI0.CAM1

But it still fails, with the same error you were getting before:

djrscally@djrscally-MIIX-510-12ISK:~/Coding/miix-510-cameras$ sudo cam -c1 -C
[0:05:04.467147383] [4011]  INFO Camera camera_manager.cpp:287 libcamera v0.0.0+1735-54ca4696
[0:05:04.491854149] [4012]  INFO IPU3 ipu3.cpp:813 Registered Camera[0] "\_SB_.PCI0.CAM1" connected to CSI-2 receiver 1
Using camera \_SB_.PCI0.CAM1
[0:05:04.495407683] [4011]  INFO Camera camera.cpp:793 configuring streams: (0) 1280x720-NV12
[0:05:04.496246145] [4012] ERROR MediaDevice media_device.cpp:802 /dev/media1[ipu3-imgu]: Failed to setup link: Invalid argument
Failed to configure camera

So time for some more debugging! A bit of progress at least though :+1:

[kitakar5525]

Glad to hear that at least camera got detected by libcamera!

Regarding DSDT, I noticed that CAM0, CAM1, PMI0, and PMI1 is not in I2C? scope. I also own Acer Switch Alpha 12 and have similar situation. This may be a problem as I can't get connection working by modding DSDT on the SA12 yet (or simply I'm doing wrong). Sorry, I can't help you regarding DSDT any further for now.

However, jhand2's surface_camera is working just fine on the SA12, too.

EDIT: updated DSDT mods anyway: https://github.com/kitakar5525/surface-ipu3-cameras/tree/lenovo_miix_510_dsdt/dsdt-mods/lenovo_miix_510

EDIT_2: (changed branch name later): https://github.com/kitakar5525/surface-ipu3-cameras/tree/lenovo_miix_510/dsdt-mods/lenovo_miix_510

[kitakar5525]

(sorry, wrong button)

[djrscally]

OK, the errors along this line:

[0:05:04.496246145] [4012] ERROR MediaDevice media_device.cpp:802 /dev/media1[ipu3-imgu]: Failed to setup link: Invalid argument
Failed to configure camera

Are because the program is trying to setup a media link that is flagged as immutable, and this is not allowed according to the documentation:

The only configurable property is the ENABLED link flag to enable/disable a link. Links marked with the IMMUTABLE link flag can not be enabled or disabled.

If we check the flags set against the link to ensure we don't call the ioctl for links flagged as immutable, we move past that problem:

diff --git a/src/libcamera/media_device.cpp b/src/libcamera/media_device.cpp
index de18d57..007a45b 100644
--- a/src/libcamera/media_device.cpp
+++ b/src/libcamera/media_device.cpp
@@ -794,7 +794,7 @@ int MediaDevice::setupLink(const MediaLink *link, unsigned int flags)
    linkDesc.sink.index = sink->index();
    linkDesc.sink.flags = MEDIA_PAD_FL_SINK;

-   linkDesc.flags = flags;
+   linkDesc.flags = flags | (link->flags() & MEDIA_LNK_FL_IMMUTABLE);

    int ret = ioctl(fd_, MEDIA_IOC_SETUP_LINK, &linkDesc);
    if (ret) {
-- 
2.17.1

But, it still doesn't take a picture:

djrscally@djrscally-MIIX-510-12ISK:~/Applications/libcamera$ cam -c1 -C
[3:46:03.326094303] [16274]  INFO Camera camera_manager.cpp:287 libcamera v0.0.0+1735-54ca4696-dirty (2020-08-24T14:19:42+01:00)
[3:46:03.365261239] [16275]  INFO IPU3 ipu3.cpp:813 Registered Camera[0] "\_SB_.PCI0.CAM1" connected to CSI-2 receiver 1
Using camera \_SB_.PCI0.CAM1
[3:46:03.369339025] [16274]  INFO Camera camera.cpp:793 configuring streams: (0) 1280x720-NV12
Capture until user interrupts by SIGINT
[3:46:03.492060791] [16275] ERROR V4L2 v4l2_videodevice.cpp:1441 /dev/video1[cap]: Failed to queue buffer 0: Broken pipe
[3:46:03.532480792] [16275] ERROR V4L2 v4l2_videodevice.cpp:1441 /dev/video1[cap]: Failed to queue buffer 1: Broken pipe
[3:46:03.573777288] [16275] ERROR V4L2 v4l2_videodevice.cpp:1441 /dev/video1[cap]: Failed to queue buffer 2: Broken pipe
[3:46:03.618320558] [16275] ERROR V4L2 v4l2_videodevice.cpp:1441 /dev/video1[cap]: Failed to queue buffer 3: Broken pipe
^CExiting

I think that the yavta tool also actually gets stuck at this same point, so teasing this out will be the next step. Also I guess we should ask the libcamera guys whether or not attempting to set up an immutable link is ever a valid operation; it might be that the links need changing to not have the flag rather than just ignoring it with my hack.

EDIT: Turns out my hack is exactly what the LinuxTV guys are doing here in their media-ctl tool that meant we didn't see this error using that to set up the link. So, this is the right way to go, and I'll probably fix that patch up a little and send it in to libcamera

[djrscally]

djrscally@djrscally-MIIX-510-12ISK:~/Coding/miix-510-cameras$ cam -c1 -C5 --file=output/frame-#.bin
[0:07:29.744777335] [4983]  INFO Camera camera_manager.cpp:287 libcamera v0.0.0+1745-28b6604f
[0:07:29.789257686] [4984]  INFO IPU3 ipu3.cpp:813 Registered Camera[0] "\_SB_.PCI0.CAM1" connected to CSI-2 receiver 1
Using camera \_SB_.PCI0.CAM1
[0:07:29.794383423] [4983]  INFO Camera camera.cpp:793 configuring streams: (0) 1280x720-NV12
Capture 5 frames
fps: 0.00 stream0 seq: 000208 bytesused: 1382400
fps: 25.00 stream0 seq: 000209 bytesused: 1382400
fps: 23.81 stream0 seq: 000210 bytesused: 1382400
fps: 24.39 stream0 seq: 000211 bytesused: 1382400
fps: 24.39 stream0 seq: 000212 bytesused: 1382400
djrscally@djrscally-MIIX-510-12ISK:~/Coding/miix-510-cameras$ ls output
frame-stream0-000208.bin  frame-stream0-000209.bin  frame-stream0-000210.bin  frame-stream0-000211.bin  frame-stream0-000212.bin

:D

Not sure how to make them into a viewable format yet though. It's midnight, tomorrow's problem. I was missing a link frequency control in my driver; you guys may have already added it to yours:

Add link frequency control

```patch diff --git a/ov2680.h b/ov2680.h index b0a0c30..ef4d4f5 100644 --- a/ov2680.h +++ b/ov2680.h @@ -44,6 +44,51 @@ struct reg_value { u8 val; }; +struct ov2680_reg_list { + u32 num_of_regs; + const struct reg_value *regs; +}; + +struct ov2680_link_freq_config { + u32 pixel_rate; + const struct ov2680_reg_list reg_list; +}; + +static const struct reg_value mipi_data_rate_840mbps[] = { + {0x0300, 0x04}, + {0x0301, 0x00}, + {0x0302, 0x84}, + {0x0303, 0x00}, + {0x0304, 0x03}, + {0x0305, 0x01}, + {0x0306, 0x01}, + {0x030a, 0x00}, + {0x030b, 0x00}, + {0x030c, 0x00}, + {0x030d, 0x26}, + {0x030e, 0x00}, + {0x030f, 0x06}, + {0x0312, 0x01}, + {0x3031, 0x0a}, +}; + +#define OV2680_LINK_FREQ_422MHZ 422400000 +#define OV2680_LINK_FREQ_422MHZ_INDEX 0 + +static const struct ov2680_link_freq_config link_freq_configs[] = { + { + .pixel_rate = (OV2680_LINK_FREQ_422MHZ * 2 * 2) / 10, + .reg_list = { + .num_of_regs = ARRAY_SIZE(mipi_data_rate_840mbps), + .regs = mipi_data_rate_840mbps, + } + } +}; + +static const s64 link_freq_menu_items[] = { + OV2680_LINK_FREQ_422MHZ +}; + static const char * const ov2680_supply_names[] = { "CORE", "ANA", @@ -92,6 +137,7 @@ struct ov2680_ctrls { struct v4l2_ctrl *hflip; struct v4l2_ctrl *vflip; struct v4l2_ctrl *test_pattern; + struct v4l2_ctrl *link_freq; }; /* GPIO Mapping for the camera */ -- 2.17.1 ```

[kitakar5525]

Wow! can you see images when you run sudo qcam? When USB cameras are connected, the command shows images. So, I guess the same thing will happen with ipu3 sensors.

[djrscally]

Yep, albeit an extremely bad one:

It's dark everywhere but the window. That could be something to do with my driver code, or it could be the image processing routines I guess. I'll have to investigate some more.

EDIT: I think fiddling with the driver code has improved it somewhat actually, I can get a visible selfie now:

So I'm going to spend some time double checking my driver code to see if I can improve it further.

[archseer]

Oh wow, great progress! I think it's likely the image processing, I think qcam might expose it, but the libcamera API layer has a programmatic access to some controls:

Controls in libcamera can be set on a per-frame basis, hardware permitting. These controls can include exposure time, focus settings, white balance, etc. This is not a useful feature for applications like video conferencing, Pinchart said, but it's important for tasks like face recognition or machine vision, where the application needs to know the parameters associated with each frame.

[kitakar5525]

@djrscally Hmm, I still can't get libcamera working... When I applied patch from https://github.com/linux-surface/linux-surface/issues/91#issuecomment-679124112, sudo cam -c1 -C causes system freeze. No kernel log left.

Have you encountered this kind of system freeze?

[djrscally]

@djrscally Hmm, I still can't get libcamera working... When I applied patch from #91 (comment), sudo cam -c1 -C causes system freeze. No kernel log left.

Have you encountered this kind of system freeze?

Nope, never. Can you reboot and try journalctl -b -1, should show the dmesg from the frozen boot

[kitakar5525]

Unfortunately, nothing was recorded after the freeze (journalctl -b -1). I'll try more.

[djrscally]

Try booting with the boot option debug instead of quiet perhaps? Or run cam under strace and hope it captures something informative?

[djrscally]

Brightness issues seems to have been because the existing ov2680 driver (which I based mine on) has auto-gain functions built in...and the ov2680 doesn't actually have an auto-gain according to its datasheet. Stripping that out and forcing it to rely on manual gain setting makes the brightness seem more or less ok to me, though the colour balance is still out of whack:

So there's a good chance that the colour balance problem is also specific to my driver, and the ipu3 stuff might be all ok.

EDIT: Bizarrely, I commented out all the gpio, clock and regulator stuff and as far as I can tell it's all still working fine. That doesn't really seem right...If the camera chip isn't consuming any of those resources...why does it even have the tps68470 at all!?

[kitakar5525]

Try booting with the boot option debug instead of quiet perhaps? Or run cam under strace and hope it captures something informative?

Thanks for the advice! but I still can't figure out what's wrong.

EDIT: Bizarrely, I commented out all the gpio, clock and regulator stuff and as far as I can tell it's all still working fine. That doesn't really seem right...If the camera chip isn't consuming any of those resources...why does it even have the tps68470 at all!?

This is interesting. So, what we actually need might be

• making connections between sensors and ipu3-cio2 (surface_camera driver with swnode patches or DSDT-overriding)
• controlling GPIOs defined in PMIC _CRS (already done in our sensor driver side or gpioset)
• some tweaks for libcamera (https://github.com/linux-surface/linux-surface/issues/91#issuecomment-679124112)

[djrscally]

Try booting with the boot option debug instead of quiet perhaps? Or run cam under strace and hope it captures something informative?

Thanks for the advice! but I still can't figure out what's wrong.

Shame; do you get the freeze running jhand's camtest.sh? That uses media-ctl and yavta rather than the libcamera code, but media-ctl does exactly what my patch does so the behaviour ought to be the same.

This is interesting. So, what we actually need might be

• making connections between sensors and ipu3-cio2 (surface_camera driver with swnode patches or DSDT-overriding)
• controlling GPIOs defined in PMIC _CRS (already done in our sensor driver side or gpioset)
• some tweaks for libcamera (#91 (comment))

Yeah pretty much, and the sensor drivers themselves of course. On that note I was planning on spending tonight tweaking the surface_camera driver so we can just enumerate devices we know are supported, and have it scan ACPI for them all, parse SSDB and create the connection rather than hard coding it into there. Did you say that you had some problems if you tried connecting both your cameras at once though?

[kitakar5525]

Did you say that you had some problems if you tried connecting both your cameras at once though?

It later turned out that libcamera won't recognize all the connections until all the sensor drivers are loaded (https://github.com/kitakar5525/surface-ipu3-cameras/issues/1).

Surface Pro/Book/Go has actually three cameras (Rear/Front/IR) but currently no one wrote the sensor driver for ov8865 (rear cam). I'll try to make it by deriving from ov8865 driver for atomisp (ipu2). Until then, we need to disable connection between ov8865 and ipu3-cio2.

[djrscally]

Oh; that's annoying. Particularly for future; if they use a different sensor in a future version that means no cameras will work until they've all got functioning sensor drivers.

Perhaps we could bodge it by having surface_camera detect whether or not a particular sensor has a driver loaded, and if not bind it to a dummy driver somehow. Have to be careful to make sure that doesn't prevent a legitimate sensor driver from grabbing the device of course.

EDIT: Actually, it should be fine I guess. If we're enumerating and connecting only supported devices, we just don't add them to the supported list until we have a driver ready and all should be well

[kitakar5525]

Shame; do you get the freeze running jhand's camtest.sh? That uses media-ctl and yavta rather than the libcamera code, but media-ctl does exactly what my patch does so the behaviour ought to be the same.

Too weird, sudo media-ctl -d /dev/media0 -p causes freeze even when no sensor/surface_camera drivers are loaded. How is your result, everyone? (no kernel patches needed to test)

[kitakar5525]

Surface Pro/Book/Go has actually three cameras (Rear/Front/IR) but currently no one wrote the sensor driver for ov8865 (rear cam). I'll try to make it by deriving from ov8865 driver for atomisp (ipu2). Until then, we need to disable connection between ov8865 and ipu3-cio2.

Added: https://github.com/kitakar5525/surface-ipu3-cameras/tree/master/ov8865 At least the driver passes probe and can be detected by libcamera with surface_camera or DSDT-overriding. EDIT: no idea how to support VCM device.

(I first tried to use old atomisp ov8865 from Android vendor tree but it's too old and not in good shape. Fortunately, there is a newly made driver for the sensor from kevlhop/MIPI_CSI2, I took this.)

[knard]

Too weird, sudo media-ctl -d /dev/media0 -p causes freeze even when no sensor/surface_camera drivers are loaded. How is your result, everyone?

the computer freeze for me too (SP6).

[djrscally]

That's probably something that needs to be logged with the LinuxTV guys then I guess...they might know about it already. Can't see anything about it on Bugzilla though.

[tarmack]

Too weird, sudo media-ctl -d /dev/media0 -p causes freeze even when no sensor/surface_camera drivers are loaded. How is your result, everyone? (no kernel patches needed to test)

I get the hang as well on the Surface Pro 2017 (SP5).

[kitakar5525]

@knard @tarmack Thanks for confirming!

I observed the hang on Arch Linux with 5.8.3 kernel. Now, I tried on Ubuntu 20.04 with 5.8.4 kernel (with surface patches and swnode patches). Somehow no hang here.

Then, with the following drivers/patch, I got noise images from qcam when I brought the smartphone's flashlight closer lol

• ov5693 from jhand2 (mine not working yet)
• surface_camera from jhand2
• using libcamera with patch from djrscally (https://github.com/linux-surface/linux-surface/issues/91#issuecomment-679124112)

In dmesg log, there are a lot of messages like the following:

[ 2104.636037] kernel: ipu3-cio2 0000:00:14.3: buffer length is 4845568 received 3604224
[ 2104.636056] kernel: ipu3-cio2 0000:00:14.3: CSI-2 receiver port 1: frame sync error
[ 2212.477038] kernel: ipu3-cio2 0000:00:14.3: CSI-2 receiver port 1: DPHY synchronization error

[djrscally]

Woohoo! This is definitely looking like progress then! I'm using Ubuntu with 5.8 kernel too, so possibly it's something their patches fix already? I'm not familiar with Arch, but istr it's a vanilla kernel there?

[kitakar5525]

Yes, Arch's kernel is almost vanilla (sometimes applied some patches).

Next interest is, does libcamera already provide a working v4l2 compat layer to use with normal apps? I see some some stuff here: https://git.linuxtv.org/libcamera.git/tree/src/v4l2

[kbingham]

@djrscally Heya, nice to see libcamera helping there. The green tint is very likely due to the complete lack of AWB/AE in our IPU3 support currently.

If you guys have interest in helping us out there, then do get in touch (irc.freenode.net #libcamera) or on our mailing list. There are two approaches to take, - we can aim to integrate the closed source binary algorithm modules that are used in ChromeOS, or we can start writing our own 3a algorithms as open source.

The good news, is that we have a fully open-source set of 3a algorithms for the Raspberry Pi - I don't yet know how much work it would be to map those onto the IPU3 - but if you guys have devices, and time to play - this is probably where I'd start.

I'm very happy to see that libcamera is starting to help real people with their devices. There's still a way to go though!

[kbingham]

Yes, Arch's kernel is almost vanilla (sometimes applied some patches).

Next interest is, does libcamera already provide a working v4l2 compat layer to use with normal apps? I see some some stuff here: https://git.linuxtv.org/libcamera.git/tree/src/v4l2

I've seen that used for real with Firefox running a camera through libcamera, so it does work. The annoyance is you have to LD_PRELOAD it which is a bit of a pain, but I know paul would love to hear back from more testing use cases. And any idea we can take to make it more user-friendly.

Of course the next goal would also be to add native libcamera support to Chrome and Firefox browsers, so that they can be detected directly.

[kbingham]

@kitakar5525 What formats are the camera sensor outputting? That looks like there is a big discrepency between what the camera sensor is transmitting, and what the rest of the pipeline is expecting.

[djrscally]

@djrscally Heya, nice to see libcamera helping there. The green tint is very likely due to the complete lack of AWB/AE in our IPU3 support currently.

If you guys have interest in helping us out there, then do get in touch (irc.freenode.net #libcamera) or on our mailing list. There are two approaches to take, - we can aim to integrate the closed source binary algorithm modules that are used in ChromeOS, or we can start writing our own 3a algorithms as open source.

The good news, is that we have a fully open-source set of 3a algorithms for the Raspberry Pi - I don't yet know how much work it would be to map those onto the IPU3 - but if you guys have devices, and time to play - this is probably where I'd start.

@kbingham ah neato, thanks for the pointer. I'm just working on getting the surface camera driver that connects the sensors and cio2 to work automagically instead of hard-coding things, but then I'd like to take a look at that, it looks pretty interesting!

[kitakar5525]

@kbingham

I've seen that used for real with Firefox running a camera through libcamera, so it does work. The annoyance is you have to LD_PRELOAD it which is a bit of a pain, but I know paul would love to hear back from more testing use cases. And any idea we can take to make it more user-friendly.

Thanks for the info! now at least sensor is recognized by firefox:

# build libcamera with v4l2 compat layer library
meson build -Dv4l2=true --reconfigure
ninja -C build

# to access /dev/media? without sudo, add user to video group:
sudo gpasswd -a __your_username__ video

LD_PRELOAD=__path_to_libcamera__/build/src/v4l2/v4l2-compat.so firefox

I can't get (that noise) image showing in firefox yet, I'll try more.

@kitakar5525 What formats are the camera sensor outputting? That looks like there is a big discrepency between what the camera sensor is transmitting, and what the rest of the pipeline is expecting.

If I understand your question correctly, format.code is set to MEDIA_BUS_FMT_SBGGR10_1X10 on probe().

• https://github.com/jhand2/linux/blob/ab18d10e4320c978c7491c697c137b3447d9536b/drivers/media/i2c/ov5693.c#L636

[jhand2]

@kitakar5525 this is awesome! One thing to note about my driver is that the controls are very incomplete. For example, you can see ov5693_s_ctrl is mostly just a stub.

It's also possible that the pll configuration is incorrect (the datasheet doesn't have settings for 19.2 MHz). This will likely result in an incorrect clock speed being reported to the CIO2 device. I'm not sure how to figure out the correct PLL config (the DS just says to contact an OmniVision support rep, so that's a dead end it seems).

[GrayHatter]

If you can find one, you should email them. I've found asking engineers when asked directly will go out of their way to provide help.

On August 27, 2020 12:47:27 PM EDT, Jordan notifications@github.com wrote:

@kitakar5525 this is awesome! One thing to note about my driver is that the controls are very incomplete. For example, you can see ov5693_s_ctrl is mostly just a stub.

It's also possible that the pll configuration is incorrect (the datasheet doesn't have settings for 19.2 MHz). This will likely result in an incorrect clock speed being reported to the CIO2 device. I'm not sure how to figure out the correct PLL config (the DS just says to contact an OmniVision support rep, which is lame).

[jhand2]

@GrayHatter The reason I say it is probably a dead end is that you technically need to sign an NDA to see OV datasheets. The one folks on this thread have been using (posted above) was leaked on some random NVIDIA forum post so none of us signed an NDA to get it.

That said, if someone has time and would like to reach out to OV it certainly is worth a try :smile:

[kbingham]

There either is or was a driver for this under the original staging for atomisp which looks like it was more featureful.

https://elixir.bootlin.com/linux/v4.12/source/drivers/staging/media/atomisp/i2c/ov5693/ov5693.c

Perhaps it was pulled out and not re added when Mauro put back the atomisp..

There might be some useful references in there though?

[jhand2]

@kbingham I looked into that and it was definitely helpful, but the input clock (XVCLK) for that system is different from the surface machines, so the PLL config is different.

However, everything I know about clocks, PLL, etc I learned in the last few months trying to work on this, and even then my understanding is tenuous. So someone with more experience wants to take a look, you may find something that I didn't.

[djrscally]

@GrayHatter The reason I say it is probably a dead end is that you technically need to sign an NDA to see OV datasheets. The one folks on this thread have been using (posted above) was leaked on some random NVIDIA forum post so none of us signed an NDA to get it.

That said, if someone has time and would like to reach out to OV it certainly is worth a try smile

I tried it already; they told me it wasn't financially worth their time to take me through the NDA process that would allow them to talk to me, and to go look online for the info I need; though I was trying through official channels rather than trying to find a sympathetic engineer online or something.

The clock and power config is interesting here tbh; I've not seen a driver that doesn't include some features to control those things...and yet as far as I can tell they're not doing anything in mine. I removed them completely, and it doesn't seem to change anything.

[GrayHatter]

I tried it already; they told me it wasn't financially worth their time to take me through the NDA process that would allow them to talk to me, and to go look online for the info I need; though I was trying through official channels rather than trying to find a sympathetic engineer online or something.

Yeah, never talk to the people who's job it is to tell you to fuck off. I usually go through linkedin looking for engineers that don't have management nor project listed as part of their job title.

[kitakar5525]

It's also possible that the pll configuration is incorrect (the datasheet doesn't have settings for 19.2 MHz). This will likely result in an incorrect clock speed being reported to the CIO2 device. I'm not sure how to figure out the correct PLL config (the DS just says to contact an OmniVision support rep, so that's a dead end it seems).

Ooooh, I just uncommented PLL setting lines and now I can see sane images. Still too dark like those pictures in https://github.com/linux-surface/linux-surface/issues/91#issuecomment-679833615, though.

EDIT: also, the image is upside down. I saw libcamera commit regarding rotation before. I'll try to find next.

[djrscally]

It's also possible that the pll configuration is incorrect (the datasheet doesn't have settings for 19.2 MHz). This will likely result in an incorrect clock speed being reported to the CIO2 device. I'm not sure how to figure out the correct PLL config (the DS just says to contact an OmniVision support rep, so that's a dead end it seems).

Ooooh, I just uncommented PLL setting lines and now I can see sane images. Still too dark like those pictures in #91 (comment), though.

OK so for me the problem there was that the sensor driver provided a V4L2_CID_AUTOGAIN control, but actually my sensor doesn't have any autogain functionality. I removed the case V4L2_CID_AUTOGAIN entry from my ov2680_s_ctrl() function and removed that handler from the v4l2 subdevice initialisation, and that fixed the brightness

This change right here and also this one here

Edit: In my case the sensor had no autogain, and so I had to remove those controls. In your case it looks like there is no gain setting control in that driver code yet. I added printouts to all my sensor's driver functions; setting the gain is one of the few things libcamera calls in terms of driver controls. It basically went > init_cfg > set_fmt > set_gain > enable_streaming.

So yeah; try adding a gain control would be my guess.

[kitakar5525]

There either is or was a driver for this under the original staging for atomisp which looks like it was more featureful.

https://elixir.bootlin.com/linux/v4.12/source/drivers/staging/media/atomisp/i2c/ov5693/ov5693.c

So yeah; try adding a gain control would be my guess.

Indeed, I see there are references to gain in that old atomisp code. The required stuff might be there.

[djrscally]

I guess it's the writes to these registers - but you'd have to trial it. In my case, libcamera never called the V4L2_CID_EXPOSURE control, so you might have to expose that register write under a control flagged as either V4L2_CID_GAIN or V4L2_CID_ANALOGUE_GAIN

And yeah, no autogain for you either:

There is no auto module in this device so this bit should always be 1

The world-facing cameras look much more complex, auto gain, exposure and focus and whatnot. So those might be more work, but also more fun.

[djrscally]

@kitakar5525 scratch that last! it does call the V4L2_CID_EXPOSURE ctrl after all! Apologies.

[kbingham]

@kitakar5525 scratch that last! it does call the V4L2_CID_EXPOSURE ctrl after all! Apologies.

Oh, I'm surprised - I suspect that might be v4l2-core setting a default value or such ... because I don't think libcamera has any interaction with the controls on IPU3 yet.

[djrscally]

@kitakar5525 scratch that last! it does call the V4L2_CID_EXPOSURE ctrl after all! Apologies.

Oh, I'm surprised - I suspect that might be v4l2-core setting a default value or such ... because I don't think libcamera has any interaction with the controls on IPU3 yet.

Could be, although I could just be mistaken and it's being peripherally called by my driver code or something. I'll take some time and check for certain.

I posted this once already and then deleted because I thought I found a bug, but it turned out to be because I rushed through code for my second sensor trying to test them both together and screwed that up! I tweaked @jhand2's surface_driver code to a) discover connection info from SSDB instead of being hard coded and b) support connecting multiple devices at the same time. It works fine for my sensor, but I only have one working driver so can't test the multiple connections thing yet. If anyone wants to try, code is here, you'll just need to add the ACPI id of your sensors to here:

static char* supported_devices[] = {
    "INT33BE",
    "OVTI2680",
};

Only add id's for drivers you have working with the existing driver for now; it still needs some work checking that the connected devices are actually owned by a driver that's initialised correctly and so on.

Also; I'm sure I messed it up somehow; if you spot anything let me know :P

EDIT: Got a semi-working driver for the other sensor now, enough to test this properly. It does auto-discover and link the sensor properly, so hurray for that. It does not link both at the same time yet; just trying to track down the problem with that.

[kbingham]

Hi, @djrscally

I think your ACPI mapping driver is looking like it could support more platforms than just the surface.

How do you feel about working towards making it more generic (or rather more specific to the CIO2 in naming rather than the surface) - and posting it for integration to the linux-kernel directly?

I'm sure there will be helpful feedback from the linux-media guys, and this would then start supporting all devices with an IPU3