"Black" Frames -- Failed to call NAPI threadsafe function on capture

[cupofnestor]

I'm attempting to draw data frames provided by the capture method:

let capture = await macadam.capture({
      deviceInfo: 0,
      displayMode: macadam.bmdModeHD1080p60,
      pixelFormat: macadam.bmdFormat8BitBGRA,
})

The issue that I am having is that the data array contains only black frames:

[0,0,0,255,0,0,0,255....]

I'm fairly certain that the configuration is correct in regards to input:

{
   ...,
   videoInputConnection: 2, // 1 << 2 = HDMI
   videoInputConversionMode: 1852796517, //none
}

I am getting an error in the node console: Failed to call NAPI threadsafe function on capture This may be related to the fact that I am using a javascript framework, but I would like to confirm . I am using .slice() to create a copy of the object inside an async function.

[rezonant]

Hello! The error you see in my experience tends to happen because your application either stops pulling new frames or is not pulling fast enough. As for the black frames, that can happen if you have not selected the appropriate display mode and pixel format for the input. These aren't the exclusive reasons for these issues though, so this may or may not apply to your situation.

[rezonant]

I have another issue open (#16) to improve handling of frame queue overruns so that it's more clear what the issue is, instead of just printing the error you are seeing.

[rezonant]

Troubleshooting tip: try using OBS DeckLink input with the configuration you are trying to use in Macadam to be sure that it's the right display mode and pixel format (without using Auto mode).

Additional tips:

• Make sure you are using await (or I guess .then() if that's what you're into) on the result of capture.frame()
• add a try/catch around it to ensure that you catch and report any rejected promise results without stopping the pull process.
• Also in my experience I've seen the alpha channel as zero and not 255 (though it looks like that is not the case for you) -- often it is wise to discard the alpha channel or force it to 255 unless you are specifically pulling an input that has an alpha channel
• If your input is really using 1080p60 this should not be a problem because that's actually 30fps interlaced (60 hz not 60 fps) but if you use a timer to pull the frames, you might be pulling too slow. Really you should just have your .frame() call in a loop with await to ensure that you are pulling frames as fast as is needed for the input.

[jesperstarkar]

What decklink driver versions are you using? <10.10 has known issues with providing black frames for a period of time after a certain uptime on the system. Don't know if that's related here though.

[cupofnestor]

Thanks guys, you've given me a lot to chew on.

The linux kernel module version is 11.2a8

[cupofnestor]

Troubleshooting tip: try using OBS DeckLink input with the configuration you are trying to use in Macadam to be sure that it's the right display mode and pixel format (without using Auto mode).

Will do, I don't see decklink input on linux though, I suppose I'll have to go look for a plugin or compile it.

Additional tips:

• Make sure you are using await (or I guess .then() if that's what you're into) on the result of capture.frame()

:rofl: I was "into" .then() until this async business came about... ...as I said before I have a feeling it may be related to the js framework (ember) I'll refactor to not use ember's databinding and see how that fares.

for ( let x = 0 ; x < frames ; x++){
      let frame = await capture.frame();
      //Object.assign(this.frame, frame);
      this.set('data',  frame.video.data.slice());
}

• add a try/catch around it to ensure that you catch and report any rejected promise results without stopping the pull process.
• Also in my experience I've seen the alpha channel as zero and not 255 (though it looks like that is not the case for you) -- often it is wise to discard the alpha channel or force it to 255 unless you are specifically pulling an input that has an alpha channel
• If your input is really using 1080p60 this should not be a problem because that's actually 30fps interlaced (60 hz not 60 fps) but if you use a timer to pull the frames, you might be pulling too slow. Really you should just have your .frame() call in a loop with await to ensure that you are pulling frames as fast as is needed for the input.

No timer, just the loop. BMD desktop video setup lists the HDMI input as 1080p60

question

Are the black frames and the NAPI errors related? I think not at this point, meaning even with the NAPI errors I should be getting correct frames... right?

[cupofnestor]

Good news, I opened up the capture in OBS and found that the pixel format ( BGRA ) caused a black screen. YUV works, so I suppose I'm off to convert BGRA=>YUV.

[sparkpunkd]

Unfortunately, as you found out, the blackmagic hardware does not do YUV -> RGBA conversions itself. If you use 10-bit YUV, you need to understand the 'V210' data packing algorithms.

In case its useful, we have another library beamcoder, a Node wrapper for FFmpeg in Node that can handle the YUV->RGBA conversions.

[rezonant]

I also have some pending work to autoconvert everything to RGBA including partially accelerated YUV->RGBA (useful when targeting HTML canvas in an electron app) that I'm hoping to land soon. More about that in #14.

Since macadam doesn't directly expose DeckLink's own conversion APIs (yet), for now beamcoder might be a good bet, you can implement simple conversions in JS but it is not really performant enough without native, and really it should be done with SIMD anyway.