search

occ-ai / obs-backgroundremoval

An OBS plugin for removing background in portrait images (video), making it easy to replace the background when recording or streaming.
GNU General Public License v2.0
2.7k stars 185 forks source link

Add simple temporal smoothing #498

Closed royshil closed 7 months ago

royshil commented 7 months ago

To handle some of the flickering that often happens with some models

royshil commented 7 months ago

@umireon What do you think?

umireon commented 7 months ago

I will try this change this week

royshil commented 7 months ago

I will try this change this week

@umireon perfect! i was hoping to release v1.1.8 very soon

umireon commented 7 months ago

I understand that this implements a low-pass filter for matt.

royshil commented 7 months ago

I understand that this implements a low-pass filter for matt.

Sort of, yes, but temporal. Really all it does is linearly blend the last mask with the current one. But since the last mask is also a blend it's an "infinite impulse response".

umireon commented 7 months ago

I'm doing z-transformation. Please wait.

umireon commented 7 months ago

I think the inverse of the current smooth factor value corresponds to the cut-off frequency and thus I suppose the value of the factor of addWeighted should be inverted on the configuration screen and it will work more straightforwardly.

royshil commented 7 months ago

I think the inverse of the current smooth factor value corresponds to the cut-off frequency and thus I suppose the value of the factor of addWeighted should be inverted on the configuration screen and it will work more straightforwardly.

the way i have it is that 1.0 corresponds to "no temporal filter" everything is taken from the current mask and 0.0 is the exact opposite. does that make sense?

umireon commented 7 months ago

the way i have it is that 1.0 corresponds to "no temporal filter" everything is taken from the current mask and 0.0 is the exact opposite. does that make sense?

It makes sense. Thank you! What I want to say in the previous comment is actually that the effect of the smooth factor on the cut-off frequency is exponential and users may want to configure the factor with the logarithmic scale because it would have an intuitive behavior.

In addition, the addWeighted operation can be done on GPU shader and it would be more efficient than OpenCV's function.

royshil commented 7 months ago

i think this cv operation on the CPU is very quick the problem with moving it to the GPU/shader is that i need to keep/save the blended image for the next frame in CPU memory... so it's not easily done (i could move the rendered blended mask... but that saves very little in time)

royshil commented 7 months ago

@umireon can you approve so i can merge it in?