Unable to get an intensity image that shows the environment similar to Ouster Studio intensity image using C++ code

[arunumd]

Describe your question

The Ouster Studio has the feature of viewing the 2D intensity image on the screen. This image looks very realistic. It shows the environment of the LiDAR like a photograph. So I am trying to reproduce the same kind of image through some C++ and OpenCV code by following the docs, provided examples and inventing some more things of my own. The code below is what I have produced for performing this operation.

    Eigen::Array<uint32_t, -1, -1, Eigen::RowMajor> reflectivity;

    if (info.format.udp_profile_lidar ==
        sensor::UDPProfileLidar::PROFILE_LIDAR_LEGACY) {
        reflectivity = scan.field(sensor::ChanField::REFLECTIVITY);
    } else if (info.format.udp_profile_lidar ==
               sensor::UDPProfileLidar::PROFILE_RNG19_RFL8_SIG16_NIR16_DUAL) {
        reflectivity = scan.field<uint8_t>(sensor::ChanField::REFLECTIVITY)
                           .cast<uint32_t>();
    } else {  // legacy or single return profile
        reflectivity = scan.field<uint16_t>(sensor::ChanField::REFLECTIVITY)
                           .cast<uint32_t>();
    }

    //! [doc-stag-cpp-destagger]
    auto reflectivity_destaggered =
        destagger<uint32_t>(reflectivity, info.format.pixel_shift_by_row);
    //! [doc-etag-cpp-destagger]

    cv::Mat intensity_img = cv::Mat(h, w, CV_8UC3);

    for (size_t i = 0; i < h; i++) {
        for (size_t j = 0; j < w; j++) {
            intensity_img.at<char>(i, j) = reflectivity_destaggered(i, j);
        }
        std::cout << "\n";
    }
    cv::namedWindow("Intensity image", cv::WINDOW_AUTOSIZE);
    cv::imshow("Intensity image", intensity_img);
    cv::waitKey(0);
    return 0;
}

However the image I get is nothing like what I get from Ouster Studio. Please see the snapshots below for comparison.

Screenshots

What I get from Ouster Studio intensity image:

What I get from my C++ code using OpenCV and Ouster SDK:

It seems as though something is wrong with this code in terms of how I try to get the reflectivity array for the environment. I am wondering what is wrong?

Platform (please complete the following information):

• Ouster Sensor? [OS-1 128 channel Rev-7]
• Ouster Firmware Version? [3.0.1]
• Programming Language? [C++]
• Operating System? [Linux and WSL using XServer]
• Machine Architecture? [x64]
• git commit hash (if not the latest).

[arunumd]

Hello,

Any idea why does the reflectivity matrix have mostly zeros in its entries? When I std::cout the intensity_img, it looks like this. Mostly overwhelmed by zeroes. So the image looks mostly black.

[Samahu]

There might be different issues with your code:

• First I think you should start by applying auto leveling to each channel before you try to pass the values to an RGB image with uint8 per channel.
  • You can find information on auto leveling and how to apply in this Google Colab Notebook.
• Second if the range of values for the reflectivity channel goes beyond 255 so you may need to re-scale or re-map the values into the range of [0, 255] before you copy the values into a uint8 otherwise the value will be truncated. You can check that by searching for the maximum value in the reflectivity image.
• Lastly based on the pattern of pixels shown in the resulting images it appears there are some problem with the order of how you copy the pixels. The shared Google Colab Notebook although it is written in python but can provide with guidance on how to a channel to an OpenCV Image or cv::Mat object.

Hope this helps!

[Samahu]

No response, marking the issue as done.