nan in illumination correction

[tstoeger]

/BIOL/sonas/biol_uzh_pelkmans_s5/Data/Users/Prisca/240215_siRNA_SM_TfRecycling_SimpleRestart

Channel 3: .std is nan for some pixels. Noted because this (luckily) causes crashes in module-specific GUI on some images.

.std is of nan is wrong and will lead to the inability to create illumination corrected images (after applying illumination correction a image consisting only of NaN will be processed - which appears to go through CPP!)

[ewiger]

The only case I found that illumination correction gives NaN as an output is when it gets NaN as an input.

>> NaN + 1

ans =

   NaN

Sum or any other operator for NaN times something will return NaN. One image containing one pixel as NaN is enough to produce such NaNs in the matrix.

But in such case getting NaNs is true not only for std, but also for the mean. What I don't understand is that also mean should contain NaNs. But it does not.

load('/home/share/nas/ethz-share5/Data/Users/Prisca/240215_siRNA_SM_TfRecycling_SimpleRestartE/BATCH/Measurements_batch_illcor_channel003_zstack000.mat')
>> numel(find(isnan(stat_values.std)))

ans =

   218

>> numel(find(isnan(stat_values.mean)))

ans =

     0

I will check the input data of raw images, if they are correct.

There also a few lines of code in illumination correction where the log10 transformation ("normalization") is done.

    % Log10 transform 
    image = log10(image);
    image(isinf(image)) = 0;

I will add an additional safety check for isnan.

[tstoeger]

thanks. Sounds great.

I believe that the tricky thing is that until we know the source of an error, we do not really know, if it is a rare problem, that has always been there (which independent of CPP / iBrain_Brutus). (Reassuringly, on other testpiplines, most of the recent errors/bugs originated from original CP code; Also I checked another project on iBrainBrutus and there was no nan in the .std)

[ewiger]

The fix worked:

>> numel(find(isnan(stat_values.std)))

ans =

     0

[tstoeger]

thanks. Let's keep our fingers crossed and hope that the fixed very rare problem of illumination correction statistics will solve the other unresolved errors that have been remaining in the same project / test-pipeline. - Again it appears encouraging that the main errors, that we are currently facing in testpipelines, seem to be rare bugs, which have always been present (rather than problems of migration / CPP / iBrainBrutus)

[tstoeger]

illumination correction stats are indeed correct

unfortunately: https://github.com/pelkmanslab/CellProfilerPelkmans/issues/15 still remains. also Measurements_Cytoplasm_Location is not created ( see https://github.com/pelkmanslab/iBRAIN_BRUTUS/issues/9 )