Unable to get consistent manual exposures

mkj-stonehill commented 3 weeks ago

I have an application in C# using .NET6 running on a Raspberry Pi (a Compute Module 4, in case it makes a difference). The Pi camera is in an enclosure with LEDs to light the target. I want to control the exposure so I get consistent images; however, there is a lot of variation. If I take 20 images in succession, most of them will be fine; about 25% of them will be noticeably darker (all the darker ones are similar to each other).

Normally I am capturing images to memory (so I can work with them); however, if I instead Capture to a JPEG file, when I look at the properties the "normal" images show an ISO speed of ISO-400, whereas the darker ones show ISO-320. There is no ISO setting that I've seen (there used to be one in the old MMAL-Sharp library). Perhaps if there was someway I could control that...? Maybe the Gain setting? But what are the units...?

I set manual exposure like this:

Cam.Settings.ExposureType = ExposureType.Manual;
Cam.Settings.ExposureTime = exposure / 100;

Expected behavior

All the images should be the same brightness.

Actual behavior

Most image are the same brightness; about 25% of them (although it varies) will be darker.

Versions used

Build machine (Windows 11, using Visual Studio 2022):

 .NET SDK:
 Version:   7.0.400
 Commit:    73bf45718d

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.22631
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\7.0.400\

Host:
  Version:      7.0.10
  Architecture: x64
  Commit:       a6dbb800a4

.NET SDKs installed:
  7.0.400 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.21 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.10 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.21 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.26 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.28 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.21 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.28 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.10 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

Execution Environment (Raspberry Pi CM4, Bullseye):

.NET SDK (reflecting any global.json):
 Version:   6.0.423
 Commit:    c5c9e53229

Runtime Environment:
 OS Name:     raspbian
 OS Version:  11
 OS Platform: Linux
 RID:         linux-arm
 Base Path:   /usr/local/bin/sdk/6.0.423/

global.json file:
  Not found

Host:
  Version:      6.0.31
  Architecture: arm
  Commit:       e2ca2f8a1c

.NET SDKs installed:
  6.0.423 [/usr/local/bin/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.31 [/usr/local/bin/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.31 [/usr/local/bin/shared/Microsoft.NETCore.App]

Download .NET:
  https://aka.ms/dotnet-download

Learn about .NET Runtimes and SDKs:
  https://aka.ms/dotnet/runtimes-sdk-info

Version of System.Device.Gpio package: 3.1.0

Version of Iot.Device.Bindings package (not needed if bug is in System.Device.Gpio) : 3.1.0

raffaeler commented 3 weeks ago

Could you please reproduce the issue with the native tools? https://www.raspberrypi.com/documentation/computers/camera_software.html

The point is that the Camera binding is just a wrapper over the native utilities. You can use any of the command line arguments to control the camera from the binding. Of course, if the problem cannot be solved using the command line arguments, there is no way to improve the situation from the binding.

Thanks

mkj-stonehill commented 2 weeks ago

Same result using command-line tools. Also same result if I tried ExposureType.Auto.

I did notice (after copying the images to my Windows machine, and then looking at the Details tab of the Properties for the JPG files) that the darker images had lower values for the "ISO Speed" setting. Since the camera binding doesn't have a property for ISO, I set the ISO by running this:

v4l2-ctl -c iso_sensitivity_auto=0,iso_sensitivity=4

Which got me an ISO of 800 on all the images I captured (which is actually what the old code used to do when using the MMALSharp package with Mono). That seemed to do the trick; now the images are much more consistent in brightness.

So I guess I should change this to a feature request to add an ISO property? How do I do that? Just open a new thingy?

raffaeler commented 2 weeks ago

@mkj-stonehill With regards to the binding, if the native libcamera tools can't do natively, there is not much I can do in the binding.

The following link highlight the relationship between the v412-ctl tool and the libcamera behavior, but I guess you already found it in your searches: https://forum.arducam.com/t/can-you-modify-camera-settings-while-libcamera-is-running/2945/11

With regards to the feature request for the ISO property, I am not sure this makes sense in the camera binding for a couple of reasons:

the libcamera will hopefully extend the capabilities in the next releases. Their goal is to create a new self-consistent stack to manage capturing video and still images.
you can still run v412-ctl using Process.Run or leverage the public ProcessRunner (that I specifically created to run the native libcamera utilities) to temporaly solve the issue while waiting for a specific libcamera settings.

Does this make sense to you as well?

mkj-stonehill commented 2 weeks ago

My understanding was that I had to enable enable Legacy Camera support, and that means libcamera won't work (instead, it's using the v4l driver). Have I got the wrong end of the stick? Maybe Legacy Camera Support means something different? Or, should we not be enabling that anymore?

Just when I thought I had it all figured out...

--mkj

raffaeler commented 2 weeks ago

As written in this doc libcamera and the old raspicam tools are mutually exclusive. Since the camera binding wraps the native tools, it just pick the native tool depending on the setting, as shown in the sample.

The legacy support refers to the old raspicam/raspivid tools while the new support refers to libcamera-still and libcamera-vid tools. I strongly suggest you to read the document linked above.

In general, libcamera is the one providing the better options but it may happen for some option to have a better support on the legacy stack. You should just try the two modalities and see what is the best for you, preferring libcamera as this is the newest and most supported stack.

dotnet / iot

Unable to get consistent manual exposures #2322