Can this plugin remove the background of a game card instead of a human portrait?

[Wells83]

Hi everybdoy. Its possible to use this plugin for create a virtual background on a square, for exemple a card game ?

[umireon]

@Wells83 Firstly, could you please provide a more detailed description of your desired use case along with some screenshots? The information you provide will help us in providing a more accurate answer.

[Wells83]

Yes, of course!

First of all, excuse my English, I'm French :)

Here's what I want to do, for my youtube channel, I want to inlay game cards into my main scene.

Initially, my camera will film my cards like this:

I then add green borders and chrominance overlay:

Unfortunately, in doing so, I lose the green in my cards :(

So I'd like to create a single-color virtual background that detects the outline of the cards, so that I can inlay them perfectly without losing the green they contain.

Importantly, when I don't have a card to show, the scene must be totally transparent.

Without card :

With card :

Last but not least, I obviously have several card sizes, orientations and formats:

To sum up, the objective is :

[umireon]

@royshil What do you think this suggestion?

[umireon]

@Wells83 I understand your needs as you want to remove the background of game cards based on AI. Our matt model is based on human portraits so that it cannot recognize your game cards. Is my understanding correct?

[royshil]

It makes a lot.of sense There are "segment anything" models and background removal model that are not for humans necessarily but for any object in the foreground, which I think would work for this pretty good.

Like Bria RMBG https://huggingface.co/briaai/RMBG-1.4 Which has onnx already and is quantized which means it may run close to realtime

[umireon]

@royshil Do you mean @Wells83 's demand is easily feasible?

[royshil]

Yes I don't think it's very hard to implement, honestly since we have everything set up in the plugin already, like onnxruntime and the image & models processing utility

[umireon]

The milestone is set to 1.1.11

[Wells83]

Thinking about it, I wonder if the best thing wouldn't be for it to detect the green zone and replace it with a virtual background.

That way, if there's no game card, we're fine and if there is a card, of course it has to detect the outline of the card.

I don't know if it's the same on your side or if it's more complicated?

[umireon]

I suppose detecting the green zone and detecting the object are almost the same things. Just the matt is inverted and it won't make any difference in the implementaion.

[Wells83]

I receive a notification when thé New version IS done ?

[umireon]

We will close this issue when we have done the implementation. We cannot promise that your suggestion will be implemented soon but stay tuned! 😄

[royshil]

implemented.

[Wells83]

Ok i test that tomorow ans i give you feedback

[Wells83]

If its the good parameter, its not working :(

[royshil]

@Wells83 You need the RMBG model It is only available in the 1.1.12 version. You may need to upgrade

[Wells83]

yes its the last version i use

[royshil]

@Wells83 do you see the RMBG option for the model?