Camera Intermittently returns single channel image, only solved by reboot

[nriaziat]

I'm using EasyPySpin with a Flir Blackfly camera which works properly most of the time, but occasionally upon itialization it will only return a depth 1 cv array. The only fix to this has been either to reboot the computer or sometimes by running the MATLAB image acquisition toolbox to open and close the connection. Im on Python 3.8 on Ubuntu 18.04.

[elerac]

Hi @nriaziat !

Could you show me more detail about it? For example, your minimal code and the error message (if it has). I have to reproduce the same issue.

[nriaziat]

self._cap = EasyPySpin.VideoCapture(0)
while not self._cap.isOpened():'
    pass
ret, self._full_res_frame = self._cap.read()
        self._frame = cv.resize(self._full_res_frame, None, fx=self._sf, fy=self._sf, interpolation=cv.INTER_AREA)
if not ret:
    break
gray_frame = cv.cvtColor(self._frame, cv.COLOR_BGR2GRAY)

This is the rough outline for how I initialize and get a frame. I believe the gray_frame line raises

'VScn::contains(scn)'
where
'scn' is 1

or something to that effect, complaining about a single channel image. I will confirm tomorrow and paste the exact error.

[nriaziat]

Hi @elerac, is there any additional information I can provide?

[elerac]

@nriaziat

Could you show me the model name of your camera (e.g., BFS-U3-04S2C-C, BFLY-U3-23S6C-C)? I guess you use an RGB color camera, right?

Let me show the result of the following printed info in both success and failure cases.

print(ret, self._full_res_frame.shape)

[nriaziat]

Yes, it is an RGB camera - BFS-U3-13Y3C.

on success, it returns: True (1024, 1280, 3) on failure, it returns: True (1024, 1280)

[elerac]

Hmm, it's weird.

To tell you about my experiment, I also use an RGB camera (I forget the model name, but it differs from yours). My camera always returns an image with depth 1. It means its shape is the same as your failure case, but for me, it's a success case. Because this is just a raw format image that has not applied the demosaicing process (aka debayering) yet. With the demosaicing process, I can get an RGB image.

ret, img_raw = cap.read()
img_bgr = cv2.cvtColor(img_raw, cv2.COLOR_BayerBG2BGR)

The depth of img_raw is 1, and img_bgr is 3.

Because of the above reason, I wonder why your camera returns an RGB image without demosaicing. I'll try to find out your issue in my environment. But it may keep you waiting because I'm so busy.

[nriaziat]

Interesting, in the case where the image is not demosaiced, wouldn’t we also expect to see 3x the size on both axes?

[elerac]

Does the both axes you refer to mean the column and row of an image? If so, the answer is NO. The image size (width and height) is the same before and after applying the demosaicing.

To clarify with Python code.

ret, img_raw = cap.read()
print(img_raw.shape)
>>> (1024, 1280)

then

img_bgr = cv2.cvtColor(img_raw, cv2.COLOR_BayerBG2BGR)
print(img_bgr.shape)
>>> (1024, 1280, 3)

As you see, the width and height are the same, and the demosaicing process adds a new dimension (it's 3).

[nriaziat]

Thanks for clarifying. Maybe I’ll try putting a try catch for the lack of depth 3 that does the demosaicing if necessary as a bandaid solution for now.

[nriaziat]

@elerac, going back to this, is there a way to specify the pixel format manually so it will either return BGR or Bayer consistently? I saw some functions that seemed to imply you could do this in the Spinnaker API.