PolarizedMono12Packed

[jfai2023]

Hi everyone,

I am working on a grabber using Aravis in linux. I have a DZK camera set as PolarizedMono12Packed, I am reading the payload directly and try to reconstruct the image of the 4 polarization angles (90,45,135,0). However I am not able to have a "clean" image of each polirization image. I would appreciate a feed back.

Here is the context and assumptions:

• Compiler: GCC (any)
• C++ 17.
• Aravis -> version 0.8, but I guess my issue will happen with any version has it is only used to interface with the camera.
• OpenCV any version (since 2.0), I am just using it to store the data.
• I am able to set and get informations from the camera (pixel, format, sensor size, payload size, ...).
• I am able to read the buffers properly.
• The device is started when I try to read the buffer.
• I do not receive any assertions from aravis or OpenCV
• The camera I am using is a DZK 33GX250

Assuming that I am reading the payload from the last available buffer and return it as an OpenCV's Mat container, which dimensions are: rows <- payload_size, cols <- 1, type <- CV_8UC1, depth <- CV_8U. As explained in the reference manual, when the camera is set in PolarizedMono12Packed the information of 2 pixels is stored in 3 bytes. In other word 24 bits store 8 values (2 for each polarization angle), which means 3 bits per value. In order to make this easier I used the following structure: ` struct attribute((packed)) polarized_t { uchar angle90_1:3; uchar angle45_1:3; uchar angle135_1:3; uchar angle0_1:3;

uchar angle90_2:3;
uchar angle45_2:3;
uchar angle135_2:3;
uchar angle0_2:3;

}; ` I then try to extract the information using the following function:

std::tuple<cv::Mat, cv::Mat, cv::Mat, cv::Mat> get_polarizations(const cv::Mat& input)
{

    int payload = input.rows;

    const polarized_t* packed = reinterpret_cast<const test_t*>(input.ptr());

    std::cout<<"Distance check: "<<std::distance(reinterpret_cast<const uchar*>(packed), reinterpret_cast<const uchar*>(packed + 1))<<std::endl;

    cv::Mat1b tmp = cv::Mat1b::zeros((payload * 2) / 3, 4);

    for(size_t i=0; i<tmp.rows; i+=2, ++packed)
    {
        tmp(i, 0) = packed->angle90_1;
        tmp(i, 1) = packed->angle45_1;
        tmp(i, 2) = packed->angle135_1;
        tmp(i, 3) = packed->angle0_1;

        tmp(i + 1, 0) = packed->angle90_2;
        tmp(i + 1, 1) = packed->angle45_2;
        tmp(i + 1, 2) = packed->angle135_2;
        tmp(i + 1, 3) = packed->angle0_1;
    }

    std::vector<cv::Mat> angles;

    split(tmp.reshape(4), angles);

    return std::make_tuple(angles.at(3), angles.at(1), angles.at(0), angles.at(2)); // 0, 45, 90, 135
}

However this approach only produce results such as the following: Polarization 0: Polarization 45: Polarization 90: Polarization 135: Raw data:

[TIS-Stefan]

Hello

I am sorry for my late answer. From my understanding, you do not unpack the 12bit format correctly, but you simply uses 2 bytes per pixel. I suggest to use "Polarized Mono 8" first and separate the incoming images with

void CreatePolarImages(int width, int height, _MyData *pData)
{
   if (!pData->mat_created)
   {
        int h = height / 2;
        int w = width / 2;
        for (int i = 0; i < 4; i++)
        {
            pData->grayImage[i].create( h, w, CV_8UC1);
            pData->colorImage[i].create(h , w, CV_8UC4);
            char WindowName[256];
            sprintf(WindowName, "Polarization %d", i + 1);
            cv::namedWindow(WindowName);
        }
        pData->mat_created = true;
    }
}

/////////////////////////////////////////////////////////////////////////
//
void AcquisitionCallback(ArvStream *pStream, void *pVoidData)
{
    _MyData *pMyData = (_MyData *)pVoidData;

    ArvBuffer *pBuffer = arv_stream_try_pop_buffer(pStream);

    if (pBuffer != NULL)
    {
        ArvBufferStatus Status = arv_buffer_get_status(pBuffer);
        if (Status == ARV_BUFFER_STATUS_SUCCESS)
        {
            pMyData->ImageReceived = true;

            int height = arv_buffer_get_image_height( pBuffer);
            int width = arv_buffer_get_image_width( pBuffer);

            pMyData->displayImage.create( height, width,
                                           ArvPixelFormat2OpenCVType( pBuffer )
                                           );

            size_t buffersize;                                         
            const void* bytes = arv_buffer_get_data(pBuffer, &buffersize);
            memcpy( pMyData->displayImage.data, bytes, buffersize );

            if( strcmp(pMyData->displayWindowName,"") != 0)
            {
                cv::imshow(pMyData->displayWindowName, pMyData->displayImage);
                cv::waitKey(1);
            }
            CreatePolarImages(width,height, pMyData);

            unsigned char* sourcePixel = (unsigned char*)bytes;
            unsigned char* destPixel[4];

            for (int i = 0; i < 4; i++)
            {
                destPixel[i] = pMyData->grayImage[i].data;
            }

            for (int y = 0; y < height; y += 2)
            {
                for (int x = 0; x < width; x += 2)
                {
                    *destPixel[1]++ = *sourcePixel++;
                    *destPixel[0]++ = *sourcePixel++;
                }
                for (int x = 0; x < width; x += 2)
                {
                    *destPixel[2]++ = *sourcePixel++;
                    *destPixel[3]++ = *sourcePixel++;
                }
            }

            for (int i = 0; i < 4; i++)
            {
                cv::cvtColor(pMyData->grayImage[i], pMyData->colorImage[i], cv::COLOR_BayerRG2BGR);
            }

            ShowPolarImages(pMyData);

            arv_stream_push_buffer(pStream, pBuffer);
        }
        else
        {
            if( Status == ARV_BUFFER_STATUS_TIMEOUT)
            {
                pMyData->Packettimeouts++;
                //printf("Paket timeouts %d\n", pMyData->Packettimeouts);
            }
            else
            {
                printf("%s\n",getErrormessage(Status).c_str());
            }
            arv_stream_push_buffer(pStream, pBuffer);
            pMyData->error = true;
        }
    }
    else
    {
        printf("Buffer is null\n");
    }
}

If this works, you may change to 16 bits using the "Polarized Mono 16" format.

Stefan

[jfai2023]

Hi Stefan,

Thank you for your answer. 1) More than half of the code in your example is missing. I was lucky to find the code zipped in an other issue. 2) When compile with aravis-0.8 and OpenCV 4.6, the executable stay idle, and do nothing.

That said I was able to test, and adapt your code to work with PolarizedMono8 and PolarizedMono16. Nonetheless I still want to use to PolarizedMono12Packed, what should I do to retrieve the information?

Thanks in advance.

Sincerely, Julien Fleuret

[TIS-Stefan]

Hello I saw you also asked my US colleagues, who asked me too. However, I added the sample for Mono 8 at https://github.com/TheImagingSource/Linux-tiscamera-Programming-Samples/tree/master/cpp/Extract-four-images-from-polarization-camera. There should nothing missing.

12bit packed means, that 12bits make a pixel. Then immediately the next 12bits follow in the second byte of the first pixel. So you need to extract this stream.

You have 3 bytes for two pixels and split them accordingly. The idea could be:

p1 = (WORD)*data & 0xFFF0;
p1 = p1 >> 4;
pdata++;
p2 = (WORD)*data & 0x0FFF;

data is a BYTE* pointer to the image data. But I did not try that on my own. If that does not work as expected, please let me know.

Best regards Stefan

[TIS-Stefan]

Hello Julien

I updated the above mentioned sample. It handles the mono 12bit packed and mono 16 bit video formats now. Unpacking of 12bits to 16bits is not that easy as I thought.

Stefan

[jfai2023]

Hi Stefan,

Thank you very much. It helps a lot.

Sincerely, Julien