camera ISP affected the manual calibration

[wyang22]

We are doing the manual photometric calibration on our embedded system, so far we still have two problems:

1. Change camera exposure time costs 0.5 hour to execute
2. The ISP for camera will automatically adjust the exposure time if it is over exposure, and also ISP will adjust other parameters to affect Vignette calibration.

How shall we handle these problems? Please help to advise, thanks

[NikolausDemmel]

I'm sorry, but I don't understand your questions. Please state clearly: what exactly are you trying to do? How exactly are you trying to do it? Where exactly is it failing? And what exactly do you want help with?

[CrisyC]

Hi, we have some problems with reproducing the experiment. For the inverse response calibration, it doesn't appear to be monotonic, so we think that some errors exist in our data. Therefore, we want to ask some help for capturing test data: 1)There are some image algorithms for improving the image quality in our camera,like AWB, Positional Gain Adjustment, Adaptive Local Tone Mapping ,Color Correcton and Aperture Correction. Except the gamma correction , whether the above mentioned algorithoms should be disabled?Or have you some suggestions about the camera parameter set-up, which one should be enable or which one should be disable? 2) For recording the images, range of exposure in our camera is 0.19ms up to 31ms, but we can't get the overexposed images, if we haven't the overexposed images, whether it's a factor influencing the response funtion generation? 3)Our camera is rolling shutter, does it matter? Please help.Thanks

[NikolausDemmel]

1)There are some image algorithms for improving the image quality in our camera,like AWB, Positional Gain Adjustment, Adaptive Local Tone Mapping ,Color Correcton and Aperture Correction. Except the gamma correction , whether the above mentioned algorithoms should be disabled?Or have you some suggestions about the camera parameter set-up, which one should be enable or which one should be disable?

Anything that changes the response function needs to be disabled. I don't know what all these algorithms do, but if they are "adaptive" during runtime, I'd disable them. Gain needs to also be constant, or else you need to convert it to a multiplicative factor and multiply the exposure time with it. The response calibration assumes a constant response function across all exposure times and the same for all pixels. A linear (multiplicative) term can different for different pixels, since that will be absorbed into the vignette.

2) For recording the images, range of exposure in our camera is 0.19ms up to 31ms, but we can't get the overexposed images, if we haven't the overexposed images, whether it's a factor influencing the response function generation?

You need to make sure to have sufficient pixels for the full range from 0 - 255 pixels. You can make a histogram over the pixel values of all images to check. If you can't, make sure to disable any adaptive control algorithms (gain control?), use a higher (constant) gain setting, or make sure your scene is brighter (more lights). If you still cant get a few values at the top or bottom, you can also manually correct them after calibration, i.e. by linearly interpolating between the last stable value and 0 or 255. It should not influence the result for other pixel values.

Another important factor in response calibration is that the scene brightness is really constant. Make sure that during the whole recording it does not change at all. No shadows from people walking around, no changing sunlight because of clouds moving, no 50Hz flickering of fluorescent lamps.

3)Our camera is rolling shutter, does it matter?

Not for response calibration, since there you don't move the camera and the scene is static. But vignette calibration from this repo will likely not work well. You can maybe mitigate the issue by ensuring the rolling shutter line-readout to be as fast as possible, and by moving the camera very slowly.

[NikolausDemmel]

PS: please don't cross post...

[CrisyC]

@NikolausDemmel Sorry, I'll pay attention.

[CrisyC]

@NikolausDemmel Excuse me , we still cannot get the correct result of the response function .I want to know how you to set up the camera explicitly, whether disable something? Thanks for your answers.

[NikolausDemmel]

Not sure what to tell you. We disable auto gain and auto exposure and make sure the response function is constant (for our current camera there are only 2 options afaik, either linear response or non linear, but in either case it doesn't change). We don't have any other dynamic setting spoke AWB or similar and I guess if you have those enabled your result might not be as expected of the the response function is not constant over time or not even the same for all pixels...

[CrisyC]

@NikolausDemmel

1. If AWB is enabled, whether it influences the response function? 2.Except it, i find that the camera becomes hot with the long working time, whether the higher temperature causes to produce more noise and influences the response function? 3.And the response fuction only means the function of the sensor or means a series of process of sensor,ADC, and DSP?

[NikolausDemmel]

We only work with grayscale images, so I don't know much about white balance, but I assume it effectively changes the response functions of the different color channels. So I would disable it. How do you deal with multiple color channels? I guess you should either estimate a response function for each channel individually, or have a fixed way to combine them to grayscale (in particular no changing white balance) and then calibrate a response for the grayscale image.

The sensor becoming hot might influence the noise a bit, but it would surprise me if it would change the response significantly (your images are not getting brighter or darker when the temperature changes, are they?)

The response models a series of changes I guess. In any case the response it is only determined up to a scalar factor, since you never know the true radiance of the scene. The exact image formation model that we assume is in the paper. It includes exposure time, response function and vignette. In this model the exposure time is a linear scaling of all pixels that may change over time, the vignette is a per-pixel linear scaling that is constant over time, and the response is a non-linear map for all pixels that is constant over time.

[CrisyC]

@NikolausDemmel The camera records color images, and then we will process it to be grayscale, rather than collect the grayscale image directly. I don't know whether this operation will influence the response function. So the key operation is to disnable some adaptive algorithoms, like AE, AG, AWB，adaptive local tone mapping？I think the most difference between our cameras is that your camera recording grayscale image and we not.

[NikolausDemmel]

Yes, you need to ensure that the mapping from color chanels to grayscale is constant.

AE you need to disable obviously since you need to manually set the exposure times for the calibration.

Later after calibration you can enable AE if you record the exposure for each image, but the other adaptive algorithms should be disabled.

[CrisyC]

Thanks, I'll try.