[OTHER] Behavior of colour correction matrix with AWB disabled

[CorvetteCole]

This isn't fully relevant to picamera2, but was hoping someone here would know off the top of their head. Will close this Tuesday if no response since I don't want to clutter this repo.

It is my understanding that when you take a photo with auto white balance enabled, the camera estimates the current colour temperature and then applies the colour correction matrix corresponding to that colour temperatures.

If a camera is going to be in a known static setting, say a sealed mini-laboratory in space, and the lighting conditions will be 100% fixed, what would the process be to basically "assume" a constant colour temperature and apply the same colour correction matrix regardless.

Basically, if I disable Auto White Balance, what happens? Does it still estimate the colour temperature and apply a relevant colour correction matrix? Can I specify a static colour temperature so that the same correction matrix is used every time?

The goal is reproducible images so that they can be directly compared over time to see accurate colour changes.

I've looked in here: https://datasheets.raspberrypi.com/camera/raspberry-pi-camera-guide.pdf and also the manual for this library but didn't find an answer to my question. I've also dug around a little in the libcamera source code but to no avail.

Thanks in advance, I know this is a bit of a weird question

[davidplowman]

If you disable the AWB after it's been running, it will keep the colour gains and colour matrix that it was using previously.

If you set explicit colour gain values it will choose the colour temperature that matches the gains best, and use the colour matrix for that colour temperature.

Hope that helps!

[CorvetteCole]

that is extremely helpful. Is there an easy strategy to derive the blue and green color gains for a target colour temperature? I know that math is done internally somewhere and I can find it, but if you know easily I'd appreciate it.

[davidplowman]

Easiest thing is to run the camera in the lighting you're interested in and use

picam2.capture_metadata()["ColourGains"]

This will give you the red and blue gains (green gain should be 1.0).

You can also work it out from the tuning file for the camera, let me know if you want to know how to do that.

[CorvetteCole]

That actually might be preferable. Goal is to pre-compute this ahead of time without much hardware access.

[davidplowman]

If you want to compute the colour gains ahead of time for a particular colour temperature, you need to find the ct_curve entry in the tuning file. The vector of numbers there is a set of triples consisting of colour_temperature, red_value, blue_value. These get repeated several times for different colour temperatures. To get the red gain, take the reciprocal of the red_value, ditto for blue.

The auto algorithm will also automatically interpolate between the listed colour temperatures (you'd have to do that manually) and does have seem leeway to adjust slightly for lighting that is unusually green or purple (not uncommon with some fluorescent lights).

You could also fix the colour matrix by deleting all the ones except the one you want. You could also interpolate your own and leave only that one in there.

[CorvetteCole]

That is again some more excellent information! Extremely useful and I really appreciate your time here. I will come back and ping you if I have further questions!

[CorvetteCole]

just wanted to come back and note for anybody interested in doing this themselves that the ct_curve list in the calibration file must have 2 color temperatures present or else libcamera will refuse to even recognize that the cameras exist. You can still disable AWB and specify gains to ensure it uses the CCM you want, but you will need to enter some arbitrary values in ct_curve

[cpixip]

If you disable the AWB after it's been running, it will keep the colour gains and colour matrix that it was using previously.

If you set explicit colour gain values it will choose the colour temperature that matches the gains best, and use the colour matrix for that colour temperature.

Hope that helps!

It that really the case? When I investigated that in the alpha-release, setting any color gain resulted in a correlated color temperature of 4500 K which was propagated down to all other modules. I checked the source code for this - as far as I can remember, it's directly in the awb-module. This might have changed - the behaviour described above would be actually more helpful than the default behaviour I have seen previously.

[davidplowman]

I did do some work on all this stuff quite some time ago so I do believe it to be true. Having said that, it's very easy for things to bit-rot, so let's give it a try! Here's what happened:

>>> from picamera2 import Picamera2
>>> picam2 = Picamera2()
...
>>> picam2.start()
...
>>> picam2.capture_metadata()["ColourTemperature"]
6495
>>> picam2.capture_metadata()["ColourTemperature"] # wave finger at camera, AWB changes temporarily
6263
>>> picam2.set_controls({"AwbEnable": 0})
>>> picam2.capture_metadata()["ColourTemperature"]
6496
>>> picam2.capture_metadata()["ColourTemperature"] # wave finger again!
6496

So it does appear to behave as described.

However, the second thing that I said (that if you set explicit colour gains it will choose the "closest" colour temperature) is inaccurate, I was misremembering. It only does that if you set explicit colour gains before you start the camera for the first time. Otherwise you just get the "fixed at whatever it currently has" behaviour that we saw above. Don't know if that's ideal or not... one to think about perhaps.

[CorvetteCole]

I find that strange, because I was able to change the colour gains in real time with AWB off and see the effects on a stream.

[davidplowman]

Yes, I probably wasn't clear. You can change the colour gains, but the reported colour temperature won't change, it'll stay fixed. The colour temperature does have an effect on the lens shading tables and colour matrix, but it's a second order effect compared to changing the colour gains.

[CorvetteCole]

Interesting... I'd personally consider that non-optimal behavior, since I expected it to change the color temperature as well. Maybe this would be worth filing an actual separate issue for.

[cpixip]

I think what I was referring to was the following code in awb.cpp :

void Awb::initialise()
{
    frameCount_ = framePhase_ = 0;
    /*
     * Put something sane into the status that we are filtering towards,
     * just in case the first few frames don't have anything meaningful in
     * them.
     */
    if (!config_.ctR.empty() && !config_.ctB.empty()) {
        syncResults_.temperatureK = config_.ctR.domain().clip(4000);
        syncResults_.gainR = 1.0 / config_.ctR.eval(syncResults_.temperatureK);
        syncResults_.gainG = 1.0;
        syncResults_.gainB = 1.0 / config_.ctB.eval(syncResults_.temperatureK);
    } else {
        /* random values just to stop the world blowing up */
        syncResults_.temperatureK = 4500;
        syncResults_.gainR = syncResults_.gainG = syncResults_.gainB = 1.0;
    }
    prevSyncResults_ = syncResults_;
    asyncResults_ = syncResults_;
}

Here, there is a code line setting the cct fixed to 4500 - and this was quite close to the cct the camera reported in my experiments with manual gain setting. But as well, that was a long time ago with the alpha-release of picamera2, and things might have changed now.

Having dug up this again, I noticed that there is another code section which is probably more relevant (lines 283-290) to the issue discussed (but, I am only guessing - this code is heavily undocumented and I did not examine this in detail):

    if (!isAutoEnabled() && firstSwitchMode_ && config_.bayes) {
        Pwl ctRInverse = config_.ctR.inverse();
        Pwl ctBInverse = config_.ctB.inverse();
        double ctR = ctRInverse.eval(ctRInverse.domain().clip(1 / manualR_));
        double ctB = ctBInverse.eval(ctBInverse.domain().clip(1 / manualB_));
        prevSyncResults_.temperatureK = (ctR + ctB) / 2;
        syncResults_.temperatureK = prevSyncResults_.temperatureK;
    }

It seems that in this section, cct is set after a mode change to the average cct suggested by the color gain-cct curves, which, by the way, look like this:

The dotted lines are directly taken from the current tuning file of the HQ camera, the solid lines come from a simulation of the HQ camera I came up with, utilizing published and measured filter responses of the CFA and the IR-filter used in the HQ camera. I can not yet explain the shift between experimental data (dotted, from the tuning file) and the simulated camera (solid lines) noticeable in the above graph. But basically, it leads to a somewhat warmer image in the simulation approach compared to the libcamera tuning file approach, the later tending to be more on the cooler looking side of things.

[CorvetteCole]

Hey there @davidplowman , we're getting some slightly strange behavior. We set up the camera with AwbMode off, static colour gains, as well as a static exposure time and analogue gain. Both cameras (on separate interfaces) are in identical lighting environments and same sensor (so same calibration file).

But across the two different interfaces we still somehow see a colour difference between them. Looking at the logs, we can see the metadata printed for interface 1:

{'SensorTimestamp': 1405640570000, 'ScalerCrop': (1000, 752, 1280, 960), 'DigitalGain': 1.0002144575119019, 'ColourGains': (1.5802781581878662, 1.1420739889144897), 'SensorBlackLevels': (4096, 4096, 4096, 4096), 'AeLocked': False, 'Lux': 563.5599365234375, 'FrameDuration': 18033, 'ColourCorrectionMatrix': (1.4462014436721802, -0.3704909682273865, -0.07571109384298325, -0.568152129650116, 2.059526205062866, -0.4913734495639801, -0.276929646730423, -0.1115819588303566, 1.3885116577148438), 'AnalogueGain': 1.0, 'ColourTemperature': 6538, 'ExposureTime': 17996}

and interface 2:

{'SensorTimestamp': 1463613969000, 'ScalerCrop': (1000, 752, 1280, 960), 'DigitalGain': 1.0002144575119019, 'ColourGains': (1.5802781581878662, 1.1420739889144897), 'SensorBlackLevels': (4096, 4096, 4096, 4096), 'AeLocked': False, 'Lux': 402.1444396972656, 'FrameDuration': 18033, 'ColourCorrectionMatrix': (1.4689751863479614, -0.3400672674179077, -0.12890997529029846, -0.5742517709732056, 2.0557384490966797, -0.48148447275161743, -0.26404303312301636, -0.3659210503101349, 1.6299641132354736), 'AnalogueGain': 1.0, 'ColourTemperature': 5887, 'ExposureTime': 17996}

Have notably different values for reported ColourCorrectionMatrix and ColourTemperature. It is my understanding that this should not be the case based on the above. What am I missing here that could be causing this variance? I want to guarantee a 100% static photo taking environment between the two cameras.

For full clarity we do the below for both interfaces:

• turn off AWB
• set colour gains to (1.580278129, 1.142074006)
• set analog gain to 1.0
• set exposure time to 18000

Thank you in advance for your help!

[cpixip]

Well, here's my guess:

"Both cameras (on separate interfaces) are in identical lighting environments and same sensor (so same calibration file)." might not be really the case. Note that your Lux-estimate is quite different between the two cameras. Now, this lux-estimate is used later in the processing chain to select an appropriate prior for the awb-unit. This in turn leads to different cct-estimates for both cameras (as observed in your metadata).

At that point, another issue with the current tuning file comes into play: even for cct's close to each other, the ccm's specified in the tuning file have widely varying coefficients. Here's an example of the variation of the main red channel gain of the libcamera ccm (which gets interpolated (dashed blue line) from the few discrete entries (red dots) in the tuning file):

So for certain cct-ranges, the coefficients of the ccm vary noticeably, even for very similar cct's; any small difference in cct-estimation will be exaggerated by the ccm's encoded in the current IMX477 tuning file.

Of course, selecting manual color gains should turn off all this automatic fiddling, but my own experiments indicate that this is not the case.

[CorvetteCole]

Hmm, it seems this could actually be worth opening an issue report then. I'll wait and see if david has anything to say about it. Appreciate your comment

[CorvetteCole]

I wonder what the AwbMode "CUSTOM" does and if that could be useful here

https://libcamera.org/api-html/namespacelibcamera_1_1controls.html

[cpixip]

The whole libcamera thing is from my point of view pretty awkward. I have yet to find a good source of documentation.

My guess with respect to your above post is that the different AWB-"modes" in effect only constrain the search range of the AWB algorithm. Note in this respect the following text snippet, taken directly out of the tuning guide:

modes : A list of AWB modes giving a name (a character string) and the “lo” and “hi” range in terms of colour temperature on the CT curve that must be searched.

Another thing which I stumbled upon: the ccm for a cct of 5920 K in the IMX477 tuning file has as red channel main entry (that is, ccm[0,0]) of 1.98691, and for a cct of 9050 K the tuning file lists a value of 2.09255. These values correspond to the red dots in my above diagram. Now, the way the interpolation of ccms is done for intermediate cct-values should result in red channel main values only in this range, namely [1.99 ; 2.093].

But: the ccm libcamera values reported in your experiments yield 1.58 for a cct of 6538 K, and of 1.469 for a reported cct of 5887 K. That does not match the above expectation at all.

Now, the ccm gets also modified by a saturation function if specified. But as this function is not present in the IMX477 tuning file, it should not be used/occur to happen. So why are the reported ccm values for a specific cct so different from the ones listed in the tuning file?

That whole libcamera-thing does not make any sense to me.

[CorvetteCole]

For the record, we are using an IMX219 vs your IMX477, and a custom tuning file so that may make that comparison less meaningful. But I do agree there is some funky stuff here

[CorvetteCole]

I've been able to work around this issue temporarily by setting all the CCMs in the tuning file to the same desired matrix, preventing interpolation.

[cpixip]

Yep, that's a possibility.

[davidplowman]

Hey there @davidplowman , we're getting some slightly strange behavior. We set up the camera with AwbMode off, static colour gains, as well as a static exposure time and analogue gain. Both cameras (on separate interfaces) are in identical lighting environments and same sensor (so same calibration file).

But across the two different interfaces we still somehow see a colour difference between them. Looking at the logs, we can see the metadata printed for interface 1:

...

Have notably different values for reported ColourCorrectionMatrix and ColourTemperature. It is my understanding that this should not be the case based on the above. What am I missing here that could be causing this variance? I want to guarantee a 100% static photo taking environment between the two cameras.

For full clarity we do the below for both interfaces:

• turn off AWB
• set colour gains to (1.580278129, 1.142074006)
• set analog gain to 1.0
• set exposure time to 18000

Thank you in advance for your help!

Hi, lots of questions in my absence (I am let out occasionally for good behaviour). Anyway, let me start with the first one!

So perhaps the most obvious thing would be to set the colour gains before you start the camera. At that point you're guaranteed the same colour gains, colour temperature, ALSC correction and CCM. (Actually, ALSC has an adaptive component but you can disable that by setting "n_iter" to zero in the tuning file.) Possibly you might do better to set the exposure time and analogue gain before starting the camera too? This would also make the camera system start more quickly.

Of course there is module to module variation which will still affect you, I guess doing the above will reveal how big this is.

The idea of using a custom AWB mode isn't a bad one. However, even if you constrain this to a single value the algorithm will still do a search transversely to the curve - meaning that there will still be variation. Also, the AWB search is a two step process (a coarse search followed by a fine search), and actually the fine search (being relatively small) doesn't get constrained to the limits of the AWB mode, meaning that it will wander a little. It seems reasonable to me that this latter behaviour should be changed down in libcamera, so I'll put that on my list of things to look at.

[CorvetteCole]

For clarity, we set those controls before the camera is started already, and we still see this variance. The only solution I found was to change all the CCMs to be the same so there was no curve to search on, but this isn't ideal.

I'll try playing around with the n_iter config, thanks for being so responsive!

[davidplowman]

For clarity, we set those controls before the camera is started already, and we still see this variance. The only solution I found was to change all the CCMs to be the same so there was no curve to search on, but this isn't ideal.

The algorithms regard guessing the colour temperature from the gains as a bit desperate, so they only do it when the camera starts for the very first time (if in non-auto mode). Are you sure you're setting them in time? Otherwise I don't really see how it could fail to calculate the same numbers every time - are you able to create a simple test script that I could look at? I've given this a try and haven't so far been able to see any variability.

[CorvetteCole]

I'll try to create a minimum reproducible test script, but the gist is we create a controls object

controls = picamera2.Controls(self._picam2)
controls.AwbEnable = 0
controls.ColourGains = (1.580278129, 1.142074006)

controls.AnalogueGain = 1.0
controls.ExposureTime = 18000

create a config

still_config = self._picam2.create_still_configuration(main={"size": resolution},
                                                       controls=self._get_camera_controls())

align the config, configure the camera

self._picam2.align_configuration(still_config)
self._picam2.configure(still_config)

then finally start the camera and capture the file

self._picam2.start()
self._picam2.capture_file(image_path, name='main')
self._picam2.stop()

I will create a test script in a few when I get a chance

[CorvetteCole]

also, to clarify there is no variation between pictures taken on the same interface. But as soon as we switch to a different interface, even with the exact same code, we get the changed metadata and slightly different CCM (compared to previous interface). Again, no variation between pictures taken on the same interface.

These two interfaces should be in identical but separate lighting conditions, but there are some small differences in the surface finish of objects surrounding the target which I theorize affect the calculation of colour temperature (thus the desire to completely bypass that). Sorry for the vagueness, I can't share too much about the research we are doing!

Additionally, neither interface reports a CCM that matches the one in our tuning file.

This goes away when I apply my workaround of setting all CCMs in the tuning file to the same matrix (presumably because the curve becomes more of a straight line haha)

[cpixip]

Setting a single ccm in the tuning file was also my workaround. I also disabled the ALSC, which is not necessary if you use a decent lens. Just screws things up. The AWB delivers in my experience most of the time better results (in comparision to a color checker) if you switch Bayes off. The algo then implements the 'gray world'-assumption, a simple algorithm which is known to fail for certain scenes. With my use cases, it performs more stable. Also, I think the contrast curve in the tuning file carries something of a personal touch - the values do not correspond to any of the standard mappings.

[davidplowman]

@CorvetteCole That's interesting about it being dependent on using a different interface. I'll try and get hold of a CM4 board so that I can try that. How are you "switching between interfaces" exactly? Are we talking about running two cameras in two different processes, or something else?

[CorvetteCole]

Nothing that complex, simply destroying the picamera2 object and recreating it on the new interface, then setting the config as aforementioned. Same process, two cameras. Both IMX219, on CM4. Sorry, haven't had a chance to get you a minimum reproducible script but will work on that in the next few days

[CorvetteCole]

Also worth noting that I can reproduce this on an RPi4 with dual interfaces as well as a CM4. Will get script to reproduce soon

[davidplowman]

I'm still wondering a bit what you're doing here. If it's a problem with regular Pi 4s as well, does that mean you're using some kind of bridge chip connected to the single CSI-2 receiver? We do support this (I believe), though the device tree will have to own up correctly to all the attached devices so that they can be enumerated and drivers loaded when the system boots. But anyway, if you can send me more specific info then I'll happily take a look. Thanks!

[CorvetteCole]

My wording was bad and I might've gotten confused there. Not meaning to mislead, that is probably a slightly different issue (yes muxing is involved). I think focusing on the issue present on a CM4 is the right idea. My apologies!

[CorvetteCole]

On a CM4, we have multiple cameras muxed through each interface. Think 3 cameras per interface for example. All photos taken on interface 1, no matter what camera is selected by the mux, have the same CCM. And all photos taken on interface 2, no matter what camera is selected, have the same CCM as well.

I'm not going to delete my above comment but please ignore the RPi4 comment, I do not think that is correct. Hopefully this helps clarify!

[davidplowman]

OK, no prob, I'll try it on a CM4 tomorrow. I don't think I have any bridge devices around so I won't see cameras behaving "the same" on the same CSI-2 port, but hopefully I should see different behaviour between the two.

[davidplowman]

Hi, just to update you on where I am with this. I've given it a go on a CM4 board with one imx219 plugged into each of the two camera ports. The code I'm running is:

def run_camera(num=0):
    from picamera2 import Picamera2
    with Picamera2(camera_num=num) as picam2:
        controls = {"ExposureTime": 60000, "AnalogueGain": 2.0, "ColourGains": (1/0.868, 1/0.428)}
        config = picam2.create_still_configuration(controls=controls)
        picam2.configure(config)
        picam2.start()
        metadata = picam2.capture_metadata()
        picam2.stop()
        picam2.close()
        return metadata

and then I randomly run either run_camera(0) or run_camera(1).

So far things all seem very well behaved and I don't see any variation in the exposure time, gains, colour matrix or reported colour temperature. However, I assume I must be doing something different so I'll probably pause for now until I can get a clearer picture of what to try next. Thanks!

[CorvetteCole]

the only functional difference to me seems to be no capturing a file, but I can't imagine that would impact it. I'll do some testing on some spare hardware and see if I can get you complete details

[CorvetteCole]

just wanted to update you and let you know that I am still working on getting these details, but I am very busy so it may take a few!

[CorvetteCole]

Hmmm, I was able to 'reproduce' it, but it may not be the bug I initially thought it was. When using switch_mode_and_capture_file, this presents itself but setting the configuration ahead of time it does not.

For example, the below code DOES reproduce the 'issue':

import picamera2
import time

def run_camera(num=0):
    image_path = f'/home/pi/{num}-TEST.jpg'
    metadata = None

    picam2 = picamera2.Picamera2(num)

    controls = picamera2.Controls(picam2)
    controls.AwbEnable = 0
    controls.ColourGains = (1/0.868, 1/0.428)
    controls.ExposureTime = 60000
    controls.AnalogueGain = 1.0

    controls = controls.make_dict()

    still_config = picam2.create_still_configuration(raw={"size": (3280,2464)}, controls=controls)
    picam2.align_configuration(still_config)
    picam2.start()
    picam2.switch_mode_and_capture_file(still_config, image_path)
    metadata = picam2.capture_metadata()
    picam2.stop()
    picam2.close()
    return metadata

print(run_camera(0))
print(run_camera(1))

The below code DOES NOT reproduce the issue:

import picamera2
import time

def run_camera(num=0):
    image_path = f'/home/pi/{num}-TEST.jpg'
    metadata = None

    picam2 = picamera2.Picamera2(num)

    controls = picamera2.Controls(picam2)
    controls.AwbEnable = 0
    controls.ColourGains = (1/0.868, 1/0.428)
    controls.ExposureTime = 60000
    controls.AnalogueGain = 1.0

    controls = controls.make_dict()

    still_config = picam2.create_still_configuration(raw={"size": (3280,2464)}, controls=controls)
    picam2.align_configuration(still_config)
    picam2.configure(still_config)
    picam2.start()
    picam2.capture_file(image_path)
    metadata = picam2.capture_metadata()
    picam2.stop()
    picam2.close()
    return metadata

print(run_camera(0))
print(run_camera(1))

The question is now I guess, is this intended behavior? I would expect the config to still be set after an image has been captured. This issue actually presents itself in an official example in the documentation for asynchronous capture (see section 6.5, page 34).

The follow code DOES reproduce the issue:

import picamera2
import time

def run_camera(num=0):
    image_path = f'/home/pi/{num}-TEST.jpg'
    metadata = None

    picam2 = picamera2.Picamera2(num)

    controls = picamera2.Controls(picam2)
    controls.AwbEnable = 0
    controls.ColourGains = (1/0.868, 1/0.428)
    controls.ExposureTime = 60000
    controls.AnalogueGain = 1.0

    controls = controls.make_dict()

    still_config = picam2.create_still_configuration(raw={"size": (3280,2464)}, controls=controls)
    picam2.align_configuration(still_config)
    picam2.start()
    picam2.switch_mode_and_capture_file(still_config, image_path, wait=False)
    # metadata = picam2.capture_metadata()
    metadata = picam2.wait()
    picam2.stop()
    picam2.close()
    return metadata

print(run_camera(0))
print(run_camera(1))

Following in that vein, capturing the metadata returned by the actual capture method also DOES reproduce this issue:

import picamera2
import time

def run_camera(num=0):
    image_path = f'/home/pi/{num}-TEST.jpg'
    metadata = None

    picam2 = picamera2.Picamera2(num)

    controls = picamera2.Controls(picam2)
    controls.AwbEnable = 0
    controls.ColourGains = (1/0.868, 1/0.428)
    controls.ExposureTime = 60000
    controls.AnalogueGain = 1.0

    controls = controls.make_dict()

    still_config = picam2.create_still_configuration(raw={"size": (3280,2464)}, controls=controls)
    picam2.align_configuration(still_config)
    picam2.start()
    metadata = picam2.switch_mode_and_capture_file(still_config, image_path)
    picam2.stop()
    picam2.close()
    return metadata

print(run_camera(0))
print(run_camera(1))

Clearly there is a bug here, but I am not sure where exactly it is. It seems to me that either the switch_mode_and_capture_file function is either not handling some configuration correctly causing incorrect CCMs, or perhaps the metadata is simply not being reported correctly. Please let me know what other information I can gather for you!

[CorvetteCole]

Another thought: this seems to correlate such that when configuration is set before the camera is started everything is all good, but if the configuration is set after the camera is started it is anyone's game. This actually makes sense when I look at one of your previous replies:

Hi, lots of questions in my absence (I am let out occasionally for good behaviour). Anyway, let me start with the first one!

So perhaps the most obvious thing would be to set the colour gains before you start the camera. At that point you're guaranteed the same colour gains, colour temperature, ALSC correction and CCM. (Actually, ALSC has an adaptive component but you can disable that by setting "n_iter" to zero in the tuning file.) Possibly you might do better to set the exposure time and analogue gain before starting the camera too? This would also make the camera system start more quickly.

Of course there is module to module variation which will still affect you, I guess doing the above will reveal how big this is.

The idea of using a custom AWB mode isn't a bad one. However, even if you constrain this to a single value the algorithm will still do a search transversely to the curve - meaning that there will still be variation. Also, the AWB search is a two step process (a coarse search followed by a fine search), and actually the fine search (being relatively small) doesn't get constrained to the limits of the AWB mode, meaning that it will wander a little. It seems reasonable to me that this latter behaviour should be changed down in libcamera, so I'll put that on my list of things to look at.

Still, it seems like pretty unexpected behavior even if there isn't a "bug" so to say

[davidplowman]

Thanks for those examples, I think that makes everything very clear. In fact it's not related to having 2 cameras, I can reproduce it easily with a single camera. It's down to the use of the switch_mode_and_capture_file method, which means the colour gains are not being set when the camera first starts.

TBH I've always been rather paranoid about reverse engineering the colour temperature from the gains because people could put in any old values, and I took the view that anything I'd calculated before was probably safer. On the other hand, you're here first and I don't know of anyone else who really cares, so I don't particularly mind changing it. It's certainly easier to explain if the colour temp always changes when you set manual gains.

So I'll post an update to the libcamera mailing list. I don't see it as at all controversial so hopefully it won't take too long to get approved.

Just as an aside, and you've probably noticed this yourself, it's much quicker to start the camera in the mode you want, do a capture, and then stop it. Starting it in the wrong mode and doing a switch mode will result in a total of 3 start/stops, which will be way slower!

[CorvetteCole]

Yes actually I've switched to setting the configuration before starting for that reason, "fixing" this issue for us anyways.

I do think prioritizing consistency is the right choice here, but maybe I am the only one who cares about this particular niche haha. This will also allow us to switch between several pre-calibrated color temperatures which is a nice feature for some of our payloads.

By the way, thanks for being so responsive. I know I've been bugging you guys across multiple issues (UDP streaming, multiple camera support, this, etc). I don't think there is anything else major I've run in to! Let me know if there is anything else I can do to help

[davidplowman]

For reference: https://patchwork.libcamera.org/project/libcamera/list/?series=3580

[cpixip]

Great! When can we expect to have that functionality available in via the usual software update?

[davidplowman]

Great! When can we expect to have that functionality available in via the usual software update?

Well, how long is an open source software project piece of string? I would honestly hope this one will be fairly quick, but ping me here again in 2 weeks if there's no news...

[davidplowman]

That patch looks to have gone through now. So I think it's just a case of us assessing which of our other (more substantial!) patches may be imminent and whether we want to wait for those.

[davidplowman]

This should now be available in the 0.3.6 release, so I'll close this issue. Please let us know if there are any problems with it!

[CorvetteCole]

Great news, I'll make sure to give it a try!

[cpixip]

I did give it a try, and as far as I can tell, the libcamera/picamera2 stuff works now as expected. Setting a manual gain changes the cct, and in turn, changes also the ccm which is used. Thank you, that is an important improvement of the new approach!

[CorvetteCole]

I can also confirm. This is really helpful to our science efforts in space!

[davidplowman]

Thanks for the confirmation!