Is the mask applied to the frame after the frame it was generated from?

[ali1234]

Pseudocode:

Thread:

• acquire lock
• process input
• write mask
• release lock

Main:

• acquire lock
• send current frame
• read mask
• apply to current frame
• release lock

Since the frame is sent and the mask is read out during the same lock period, doesn't that mean that the mask belongs to the previous frame? So if something is moving on the screen, the mask will lag one frame behind it? Is my understanding correct?

[phlash]

I chose to run mask generation on a separate thread, as some more complex models do not run at video frame rates, hence the logic you have summarised above to decouple the processing rates. For models that can run at frame rates, then yes, this results in a 1-frame lag, it's a compromise to keep the video frame rate correct (or OBS throws a fit!)

We do similar in the main backscrub app too.

[ali1234]

This will still be limited to the speed of the worker thread though right? It just buys the worker until the next frame arrives to finish processing the current one. If it can only do say 15 fps and the incoming feed is 60 fps, frames will still back up, because there is no mechanism for the main thread to skip a frame if the worker is busy?

[ali1234]

Ah no, ignore that, it unlocks while actually processing a copy of the frame.

[phlash]

@ali1234 - any objection to closing this?