Calculating the pyramid with colorSpace set to gray results in two black channels

[JN-Jones]

For a project of mine I've to work with colorSpace set to gray. While chnsCompute seems to respect that and returns only one color channel, chnsPyramid calculates only one channel but returns three channels where the second and third channel are simply black. The original matlab toolbox seems to discard those two channels, would be nice if this library could do the same.

[JN-Jones]

Little update: It seems that chnsCompute ignores this too, however if I'm inputting a grayscale image I only get one colorchannel. Looks like the number of input channels is used instead of the number of requested channels.

[headupinclouds]

https://github.com/elucideye/acf/blob/17b8da4c72290a55ecfcca7899dbdd1da061ce80/src/lib/acf/chnsCompute.cpp#L139

int Detector::chnsCompute(const MatP& IIn, const Options::Pyramid::Chns& pChnsIn, Detector::Channels& chns, bool isInit, MatLoggerType pLogger)

I'll add a few notes to help track the relevant code sections.

There is a field Field<Color> Options::Pyramid::Chns::pColor in the input structure passed to chnsCompute(...) and the input MatP image comes with a dimension: int MatP::channels().

                struct Color
                {
                    Field<int> enabled;
                    Field<double> smooth;
                    Field<std::string> colorSpace;

                    void merge(const Color& src, int mode);
                    friend std::ostream& operator<<(std::ostream& os, const Color& src);

                    template <class Archive>
                    void serialize(Archive& ar, const uint32_t version);
               };

In your case first scenario, you are passing in a 3 channel RGB image and are requesting colorSpace == "gray". It should return a single color channel, but instead you are getting 3 channels, where the first one is grayscale. We don't want the second two.

Assuming that is correct, I can try to reproduce it this weekend. Feel free to send a fix if you have one.

https://github.com/elucideye/acf/blob/1c9fa310e2d743f7b91d44c1b1016e40376eb183/src/lib/acf/chnsCompute.cpp#L222

        rgbConvert(I, I, p.colorSpace, true);

:point_up: It looks like I is use as an input and output type here. I believe the output type should be (re)allocated using the channel count of the colorspace specified by p.colorSpace:

https://github.com/elucideye/acf/blob/1c9fa310e2d743f7b91d44c1b1016e40376eb183/src/lib/acf/rgbConvert.cpp#L109

int Detector::rgbConvert(const MatP& IIn, MatP& J, const std::string& colorSpace, bool useSingle, bool isLuv)
{
    // <SNIP>
    if (!IIn.empty() && (flag != 1) && (flag != 4))
    {
        rgbConvertMex(IIn, J, flag, useSingle);
    }

https://github.com/elucideye/acf/blob/1c9fa310e2d743f7b91d44c1b1016e40376eb183/src/lib/acf/toolbox/rgbConvertMex.cpp#L385-L387

    J.create(I.size(), I.depth(), I.channels());
    float* pJ = J.ptr<float>();
    rgbConvert(const_cast<float*>(pI), pJ, I.size().area(), I.channels(), flag, float(nrm));

[JN-Jones]

In your case first scenario, you are passing in a 3 channel RGB image and are requesting colorSpace == "gray". It should return a single color channel, but instead you are getting 3 channels, where the first one is grayscale. We don't want the second two.

Exactly. If I simply change the input image from a rgb image to a grayscale image without modifying any other param I get only one color channel as output. For now I'm simply removing those two channels after converting them to std::vector<cv::Mat> but obviously that's far from efficient.

[headupinclouds]

This should be fixed in the latest version.

[JN-Jones]

Hi, sry for the late feedback: I've updated the library today and noticed that apparently there's still an issue. Scenario is still the same, I've an image which is usually a color image but can also be a grayscale image and I set colorSpace to gray. While the number of outputed channels is now correct (always 8) the computed channels seem to be wrong:

• Input color image and calculating the whole pyramid returns proper channels and scales
• Input gray image and calculating only the channels returns proper channels
• Input color image and calculating only the channels returns black images I've tested this with my own function (managing options on my own and directly calling chnsCompute) and using computeChannels where I modified the setting ( https://github.com/elucideye/acf/blob/master/src/lib/acf/ACF.cpp#L202 ).

I've debugged this a bit and noticed a few things:

• rgbConvert is called with wrong parameters in chnsCompute: https://github.com/elucideye/acf/blob/master/src/lib/acf/chnsCompute.cpp#L221 (The fourth param is useSingle which is set to true in chnsPyramid: https://github.com/elucideye/acf/blob/master/src/lib/acf/chnsPyramid.cpp#L247 )
• chnsPyramid contains an unnecessary if: https://github.com/elucideye/acf/blob/master/src/lib/acf/chnsPyramid.cpp#L231-L233
• In chnsPyramid you check for I.channels() before converting the image, however in chnsCompute you don't

However the main issue with black images is apparently caused by using the same MatP object for input and output rbgConvert(I, I, ...). When changing this to MatP t; rbgConvert(I, t, ...); I = t; everything runs normally again. Note that chnsPyramid doesn't have this issue as it already uses differenct objects for input and output.

[headupinclouds]

Okay, thanks for the update. I'll re-open this and sort through it more carefully.

[JN-Jones]

Any Update on this one? Really would like to see an official fix for this instead of my workaround.

[headupinclouds]

I've added a date for this one and will work on a fix this weekend. The notes above are helpful. For conciseness, Can you also share a small Matlab code sample to illustrate the expected behavior? Something like this

I=imread('dodecagon.png') * 255; % force [0 ... 255]
pPyramid=chnsPyramid();
P2=chnsPyramid(I,pPyramid);
splitA = num2cell(P2.data{1}, [1 2]);
P2_montage = horzcat(splitA{:});
imwrite(P2_montage, '/tmp/acf_matlab.png');

[JN-Jones]

Sure though the dodecagon seems to result in an black image anyways so I've used an image from the dataset I'm using:

I=imread('traffic.jpg');
pPyramid=chnsPyramid();
pPyramid.pChns.pColor.colorSpace = 'gray';
P2=chnsPyramid(I,pPyramid);
splitA = num2cell(P2.data{1}, [1 2]);
P2_montage = horzcat(splitA{:});
imwrite(P2_montage, '/tmp/acf_matlab.png');

[headupinclouds]

--color gray

polly.py --toolchain xcode --config Release --fwd HUNTER_CONFIGURATION_TYPES=Release --install
_install/xcode/bin/acf-pyramid --input=${HOME}/Downloads/40408926-7e6499ae-5e69-11e8-9573-3fc8f15385c8.jpg --output=/tmp --colorspace gray
montage -tile 8x -border 0 -geometry +0+0 /tmp/40408926-7e6499ae-5e69-11e8-9573-3fc8f15385c8_*_*_*.png /tmp/pyramid_gray.jpg

[headupinclouds]

--color luv

polly.py --toolchain xcode --config Release --fwd HUNTER_CONFIGURATION_TYPES=Release --install
_install/xcode/bin/acf-pyramid --input=${HOME}/Downloads/40408926-7e6499ae-5e69-11e8-9573-3fc8f15385c8.jpg --output=/tmp --colorspace luv
montage -tile 10x -border 0 -geometry +0+0 /tmp/40408926-7e6499ae-5e69-11e8-9573-3fc8f15385c8_*_*_*.png /tmp/pyramid_luv.jpg

[headupinclouds]

--color rgb

polly.py --toolchain xcode --config Release --fwd HUNTER_CONFIGURATION_TYPES=Release --install
_install/xcode/bin/acf-pyramid --input=${HOME}/Downloads/40408926-7e6499ae-5e69-11e8-9573-3fc8f15385c8.jpg --output=/tmp --colorspace rgb
montage -tile 10x -border 0 -geometry +0+0 /tmp/40408926-7e6499ae-5e69-11e8-9573-3fc8f15385c8_*_*_*.png /tmp/pyramid_rgb.jpg

[headupinclouds]

@JN-Jones: The above were created with https://github.com/elucideye/acf/pull/80 which adds --colorspace to the acf-pyramid console utility, but the actual lib updates are already merged. Let me know if that doesn't address your use case.

[JN-Jones]

Seems to be fixed, thanks