Querying supported `PixelFormat`? And issues arising

[g40]

Hello TIS

We have been lent a VGA camera (DFK 23G 618) whilst waiting delivery of some 4K versions. This is causing me some problems. Questions arising:

1. The devicePropertyMap content does not appear to be consistent with the DFK 33GX545. For example any attempt to set the image width/height fails with the following message I am aware that the VGA camera only supports a single resolution but the error message is surprising.

2. What is the correct way to query and display the current camera pixel format? Setting like this seems to work but the VGA camera image data does not render correctly. It looks like it might be still sending some YUV format?

           static constexpr ic4::PixelFormat dev_format = ic4::PixelFormat::BGR8;
           ok = map.setValue(ic4::PropId::PixelFormat, dev_format, err);

   This image is rendering what I think is 24bpp data with dimensions of 640x480:

3. Is there an IC4 function to convert an imageBuffer (whatever its format) to Windows style BGR (24bpp) or BGRA (32bpp)? All I can find are the write to a bitmap file family.

4. This camera has also shown a tendency to disappear. It is using the latest GigE and IC4 installations with the camera attached to a Netgear POE switch and connected directly to an i9 equipped PC.

Thanks again for listening. Any advice much appreciated.

[g40]

A little bit more information. Attempting to set the pixel format directly generates the following error. What is the correct enumeration for BGR mode?

    static constexpr ic4::PixelFormat dev_format = ic4::PixelFormat::BGR8;
    ok = map.setValue(ic4::PropId::PixelFormat, dev_format, err);
    s = u::sprintf("setValue: PixelFormat::BGR8 %s %s", ok ? "true" : "false", err.message().c_str());

[TIS-Tim]

Hi Jerry,

you are right, those write errors are annoying. We will think of a way to avoid them.

The image you posted looks like 8-bit data being interpreted as BGRA (32bpp).

Simplest way to get BGR8 data is to set the sink format as BGR8 (in the sink's constructor) and leave the camera's PixelFormat (which is what actually goes over the wire) as Bayer8. ic4 will then automatically convert all images to the specified pixel format.

The camera itself does not support the BGR8 format, therefore you get the error when trying to set it. A more descriptive error message might be helpful here.

If you only want to convert select images, there is a way:

// (typed into github issue, I hope this compiles...)

ic4::ImageBuffer convertToBGRA32bpp( std::shared_ptr<ic4::ImageBuffer> srcBuffer )
{
    // First, create a buffer pool to generate additional buffers
    auto pool = ic4::BufferPool::create();

    // Define the format for destination buffer
    auto destType = ic4::ImageType( ic4::PixelFormat::BGRa8, srcBuffer->imageType().width(), srcBuffer->imageType().height() );

    // Allocate the destination buffer
    auto destBuffer = pool->getBuffer(destType);

    // Copy from source to destination
    destBuffer->copyFrom( srcBuffer );

    return destBuffer;
}

Using the buffer pool looks a little cumbersome in this example, but it works great if you continuously transform buffers this way since it reuses memory if possible.

The copyFrom function does not necessarily support all possible pixel format conversions, but there usually is support converting to BGRa8.

[g40]

Hi Tim, thanks again

OK so the quick fix is your suggestion re the sink. Sadly this fails using the DFK 33GX545 as shown below. Code snip included in case I've misunderstood something essential.

// Create a QueueSink to capture all images arriving from the video capture device
sink_t qsink = ic4::QueueSink::create(nl, ic4::PixelFormat::BGRa8, err);
// Start the video stream into the sink
ok = grabber.streamSetup(qsink, ic4::StreamSetupOption::AcquisitionStart, err);
if (!ok) {
    MessageBoxA(args.hWnd, err.message().c_str(), nullptr, MB_OK);
    return;
}

[TIS-Tim]

Looks like BGR8 -> BGRa8 is the one transformation that does not work... If you set camera's PixelFormat to Bayer8, this should work (and, as a bonus, you get a higher maximum frame rate).

[g40]

Hi Tim,

FYI, the only combination I've got to work is to set the camera to use ic4::PixelFormat::BayerRG8 and to then open the imageSink like so:

        // set the camera format as per https://github.com/TheImagingSource/ic4-examples/issues/7
    ic4::PixelFormat ipf = ic4::PixelFormat::BayerRG8;
    ok = map.setValue(ic4::PropId::PixelFormat, ipf , err);

    // Create a QueueSink to capture all images arriving from the video capture device
    sink_t qsink = ic4::QueueSink::create(nl, ic4::PixelFormat::BGR8, err);

otherwise I get errors.

Does this mean various cameras support different Bayer flavours? There is surely a common default?

Kind of a nuisance, so let me circle back to the question which was:

How do I enumerate the pixel formats a camera does support?

Thanks for listening.

[TIS-Tim]

map[ic4::PropId::PixelFormat].entries() will get you a vector of ic4::PropEnumEntry which you can query for name() or displayName().

And yes, the variety of bayer pattern is camera-specific.

[g40]

Hi,

So, attempting to set a Bayer mode on the DFK 23G 618 ends up with this error message, which is not the case for the DFK 33GX545. Any thoughts? Which pixel format(s) do you think should work?

[TIS-Tim]

Like I said, camera-specific. BayerGB8 should work (0x0108000A)

[g40]

Ok, we have finally got the VGA camera to produce decent data*. I cannot recommend highly enough that an example demonstrates these two key points.

Many thanks here for the help and forbearance. Greatly appreciated.

1. How to enumerate valid pixel formats

   std::vector<ic4::PropEnumEntry> ve = map[ic4::PropId::PixelFormat].entries();
   for (auto& e : ve)
   {
   ic4::PixelFormat pf = (ic4::PixelFormat) e.intValue();
   // pick a Bayer format for the camera, let the sink convert to BGR(A)
   }

2. Detail how the sink can be used to do format conversion!

   // Create a QueueSink to capture all images arriving from the video capture device
   // since we specfied a pixel format, the sink will do the conversion for us
   sink_t qsink = ic4::QueueSink::create(nl, ic4::PixelFormat::BGR8, err);
   // Start the video stream into the sink. *Always* check returns for errors
   ok = grabber.streamSetup(qsink, ic4::StreamSetupOption::AcquisitionStart, err);

[TIS-Tim]

In a future release there will be a pseudo-pixelformat value that can be used to directly set any bayer format with a specified bit depth, regardless of the actual pattern. Enumerating formats and make decisions based on what is available is not something one should be forced to do.

[TIS-Tim]

Setting any bayer pixel format supported by the camera has been simplified in ic4 Version 1.1: PixelFormat::AnyBayer8

[g40]

Good to know. Thanks Tim.