Color order of retrieved image

[FangLinHe]

Some information first:

• Zed SDK version: 2.8.0
• Cuda version: 10.0
• Language: C++

We recently noticed some confusion of the color order of retrieved images by Camera::retrieveImage function. It seems to us the retrieved image is in RGBA format, based on:

• your document - Sharing image data between ZED SDK and OpenCV:

  // Create an RGBA sl::Mat object
  sl::Mat image_zed(zed.getResolution(), MAT_TYPE_8U_C4);
  // Create an OpenCV Mat that shares sl::Mat data
  cv::Mat image_ocv = slMat2cvMat(image_zed);

• in the sl_zed/defines.hpp, in enum VIEW, the VIEW_LEFT which is passed as a parameter of Camera::retrieveImage, also has the comment:

  VIEW_LEFT, /**< Left RGBA image. Each pixel contains 4 usigned char (R,G,B,A). sl::MAT_TYPE_8U_C4.  */

• and your example code:

  void saveSbSImage(Camera& zed, std::string filename) {
  ...
  cv::cvtColor(sbs_image, sbs_image, cv::COLOR_RGBA2RGB);
  ...
  }

• I also had a look at the headers, and even found in sl_core/utils/types.hpp:

  /**
  \class Vector4
  \ingroup Core_group
  \brief Represents a four dimensions vector for both CPU and GPU.
   */
  template <typename T>
  class /*@cond SHOWHIDDEN*/SL_CORE_EXPORT/*@endcond*/ Vector4 {
      static const int nbElem = 4;
  public:
      /// @cond
      union {
  
          struct {
              T x, y, z, w;
          };
  
          struct {
              T r, g, b, a;
          };
  
          struct {
              T ox, oy, oz, ow;
          };
          T v[nbElem];
      };
      /// @endcond
      ...
  };

  where struct { T r, g, b, a; }; indicates the pixel value order is RGBA. (This struct is used in the docstring of ERROR_CODE retrieveImage(Mat& mat, VIEW view = VIEW_LEFT, MEM type = MEM_CPU, int width = 0, int height = 0); in sl_zed/Camera.hpp, which is the returned type of sl::Mat::getValue:

  Mat leftImage, depthView; //create sl::Mat objects to store the images
  while (true) {
  // Grab an image
  if (zed.grab() == SUCCESS) { // A new image is available if grab() returns SUCCESS
      zed.retrieveImage(leftImage, VIEW_LEFT); // Get the rectified left image
      zed.retrieveImage(depthView, VIEW_DEPTH); // Get a grayscale preview of the depth map
  
      //Display the center pixel colors
      sl::uchar4 leftCenter;
      leftImage.getValue<sl::uchar4>(leftImage.getWidth() / 2, leftImage.getHeight() / 2, &leftCenter);
      std::cout << "leftImage center pixel R:" << (int)leftCenter.r << " G:" << (int)leftCenter.g << " B:" << (int)leftCenter.b << std::endl;
  
      sl::uchar4 depthCenter;
      depthView.getValue<sl::uchar4>(depthView.getWidth() / 2, depthView.getHeight() / 2, &depthCenter);
      std::cout << "depthView center pixel R:" << (int)depthCenter.r << " G:" << (int)depthCenter.g << " B:" << (int)depthCenter.b << std::endl;
  }
  }

  After testing a bit, we finally confirmed the pixel values are actually in BGRA order instead of RGBA order. However, searching in the documentations, code, issues, etc., we didn't find any information about this fact. I'd be nice if you could confirm the values are truly in BGRA order but not RGBA order and update the document to prevent similar confusion in the future. Thank you in advance for the reading and look forward to hearing from you.

[annaborn]

The same for me!

I was really irritated when I sent the image retrieved by zed camera to our Image Viewer and saw the swapped colors. After debugging I can confirm the issue described above. The memory layout of color channels in LEFT mode is not RGBA, but BGRA.

The issue was opened one year ago. But still no answer and no updates in the documentation.

@adujardin, please check this issue!

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment otherwise it will be automatically closed in 5 days