Lab colorspace support

[cceyda]

I wish Lab colorspace conversion was also supported. It is used often in neural colorization. Are there any plans for something like 'rgb2lab'?

[JanuszL]

Hi @snu-ceyda , We don't have such a plan now, but you are more than welcome to write such an operation. You na take a look how it works now here, here and here. We would be more than happy to review any PR adding this functionality.

[cceyda]

Unfortunately I'm not familiar enough with C, but It looks like it might be possible to do this referencing code from OpenCV implementation (COLOR_RGB2Lab). Would nvJPEG also have to support this or are they separate things?

[JanuszL]

@snu-ceyda - nvJPEG is a separate thing and it supports the following data formats as I see: . Yes, probably an OpenCV implementation is a good reference if anyone is eager to pick that up and submit a PR we would be more than happy to review it.

[ijpq]

I found that the color space conversion of dali is implemented by the ocv api of cvtcolor, so in fact, in order to increase rgb2lab, there is no need to do some linear operations on mat, but only need to define variable of rgb2lab and pass it to cvtcolor. Can the color conversion on the cpu side be realized by calling?

[ijpq]

I found that the color space conversion of dali is implemented by the ocv api of cvtcolor, so in fact, in order to increase rgb2lab, there is no need to do some linear operations on mat, but only need to define variable of rgb2lab and pass it to cvtcolor. Can the color conversion on the cpu side be realized by calling?

i mean there is no need to focus on a bit low level api implement such as _parallel_for__/ _cv_halcvtBGRtoBGR?

[JanuszL]

@ijpq - for CPU it sounds like a solution, for the GPU NPP supports only BGRToLab/LabtoBGR (at least that what I see) so you may need to write something on your own.

[ijpq]

@ijpq - for CPU it sounds like a solution, for the GPU NPP supports only BGRToLab/LabtoBGR (at least that what I see) so you may need to write something on your own.

My current understanding is that RGB2LAB on the CPU side can be implemented mainly by calling cvtcolor, while RGB2LAB on the GPU side needs to implement an npp api, similar to nppiYCbCrToRGB_8u_C3R, so I should learn npp

[ijpq]

I thought this problem should be simple, but I don’t know enough about color space conversion. on the CPU or GPU side, the conversion of LAB color space and other color spaces has encountered some bugs when testing as follows:

CPU side

Error when executing CPU operator ColorSpaceConversion, instance name: "ColorSpaceConversion", encountered: Error in thread 0: OpenCV(4.4.0) /opencv-4.4.0/modules/imgproc/src/color.simd_helpers.hpp:92: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<3>; VDcn = cv::impl::{anonymous}::Set<3, 4>; VDepth = cv::impl::{anonymous}::Set<0, 5>; cv::impl::{anonymous}::SizePolicy sizePolicy = (cv::impl::<unnamed>::SizePolicy)2; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]' Invalid number of channels in input image: 'VScn::contains(scn)' where 'scn' is 1

maybe i should looking into operator test detail closer.

GPU side

[/opt/dali/dali/operators/image/color/color_space_conversion.cu:242]: NPP error "NPP_STEP_ERROR"

[JanuszL]

In case of npp, I assume it comes from:

            // GRAY -> BGR
        detail::ConvertGrayToRGB8uKernel<<<grid, block, 0, stream>>>(
          input_data, output_data, total_size);
        // BGR -> Lab
        DALI_CHECK_NPP(
          nppiBGRToLab_8u_C3R(
            output_data, nStepOutput, output_data, nStepOutput, size));

nStepOutput is calculated based on

const int nStepOutput = output_C * size.width;
int output_C = NumberOfChannels(output_type_);
inline bool IsColor(DALIImageType type) {
  return type == DALI_RGB || type == DALI_BGR || type == DALI_YCbCr;
}

inline int NumberOfChannels(DALIImageType type) {
  return IsColor(type) ? 3 : 1;
}

So you need to add LabsSpace to IsColor otherwise DALI would assume it has only 1C and the nStepOutput is not calculated properly. In the case of OpenCV, I don't see anything obvious, I would run this case under GDB and examine the input data that is passed to the conversion function to see if data is really 1C/3C as it should be.

[ijpq]

about debug : i'm trying to gdb with operator_test.bin ,but telled mdb.c no such file. i think that should be obj and src separated? but i cannot found mdb.c. about mean error: RGB,BGR,YCBCR and LAB conversion's mean error is about 20. i found mean error should be smaller than eps=0.001 after normailze to 255. This requirement should be in the range of mean error of 0.1 percent, right? If this is the case, dividing by 255 when normalization may not be very suitable for the lab color space, because the value range of the L channel is 0 to 100

[JanuszL]

RGB,BGR,YCBCR and LAB conversion's mean error is about 20. i found mean error should be smaller than eps=0.001 after normailze to 255. This requirement should be in the range of mean error of 0.1 percent, right? If this is the case, dividing by 255 when normalization may not be very suitable for the lab color space, because the value range of the L channel is 0 to 100

As I understand it is an average pixel difference. In the case of this test suite CPU and GPU, implementations are compared, so even if L has a range 0-100, it should be still valid to compare.

[ijpq]

I know where I was wrong before. When converting RGB to LAB, I need to divide the pixel by 255 to map to the range of [0,1]. Now after correction, the test error is between 0.7 and 0.001 testing result as follow: [----------] 1 test from ColorSpaceConversionToBGRTest/3, where TypeParam = dali::Lab [ RUN ] ColorSpaceConversionToBGRTest/3.test /opt/dali/dali/test/dali_test_single_op.h:586: Failure Expected: (mean) <= (eps_), actual: 0.382591 vs 0.001 /opt/dali/dali/test/dali_test_single_op.h:586: Failure Expected: (mean) <= (eps_), actual: 0.740729 vs 0.001 /opt/dali/dali/test/dali_test_single_op.h:586: Failure Expected: (mean) <= (eps_), actual: 0.0976087 vs 0.001 /opt/dali/dali/test/dali_test_single_op.h:586: Failure Expected: (mean) <= (eps_), actual: 0.296535 vs 0.001 /opt/dali/dali/test/dali_test_single_op.h:586: Failure Expected: (mean) <= (eps_), actual: 0.036912 vs 0.001 /opt/dali/dali/test/dali_test_single_op.h:586: Failure Expected: (mean) <= (eps_), actual: 0.262305 vs 0.001 /opt/dali/dali/test/dali_test_single_op.h:586: Failure Expected: (mean) <= (eps_), actual: 0.145898 vs 0.001 /opt/dali/dali/test/dali_test_single_op.h:586: Failure Expected: (mean) <= (eps_), actual: 0.00117774 vs 0.001 /opt/dali/dali/test/dali_test_single_op.h:586: Failure Expected: (mean) <= (eps_), actual: 0.00318612 vs 0.001 /opt/dali/dali/test/dali_test_single_op.h:586: Failure Expected: (mean) <= (eps_), actual: 0.195745 vs 0.001

but

As I understand it is an average pixel difference. In the case of this test suite CPU and GPU, implementations are compared, so even if L has a range 0-100, it should be still valid to compare.

I don’t understand your reply very well. do you mean calculation of average pixel difference : average of sum up pixel difference of GPU and CPU computing results? In fact, I should understand this calculation method by reading the code. It should be a problem of my own ability. I still feel dizzy when reading the test code. Thank you for your patient guidance :+1:

[mzient]

If the implementations (OpenCV and NPP) are using different roundings, then a mean absolute error of 0.5 is reasonable. Look at the test converting to YCbCr - this one sets some thresholds. You should follow this implementation and set the thresholds as well.