Onboard rembg for image background removal

[db0]

Motivation I want to be able to remove image backgrounds as part of my processing.

Description The rembg python module uses a pytorch model to remove image backgrounds. It's not perfect but it's really damn good. I would like to have an option to use it during processing

[RunDevelopment]

!! This library is for Python 3.9 only !!

This might be a problem, since we support more than just 3.9.

@joeyballentine Honestly, this seems like the perfect third-party module (narrow in scope, self-contained, could be a single node). If we include it into Chainner, then we should make it an official third-party module as soon as we have an API (#786).

[joeyballentine]

The models are available for download and are just onnx models. Have you tried just running them with onnx in chainner? No need to add a third party library that only supports python 3.9 if you can just run the models already

[joeyballentine]

The only thing would be if it does operations other than just the standard image inference ones. Like if it does something to the images after or before inference. Then yeah, support would be needed to be added manually.

[db0]

Sorry I am just not advanced enough for this usage. I'll see if I can figure it out myself, but in the meantime, it would still be cool if the chaiNNer module for this existed by default.

[joeyballentine]

My main point is that we might be able to support this without needing to add this dependency. If there's something specific about the models I'd be able to detect, I could add support for it (if it doesn't already just work) like I do for pytorch with all the models there

[theflyingzamboni]

Looking at it, it looks like it does things aside from just running the models, much like that Armory thing.

[theflyingzamboni]

@db0 Have you been able to make this package work? I pip installed it, but importing the module completely breaks opencv for me. Like, the moment I add a line importing it, cv2 functions no longer do anything. The interpreter skips the commands, and a script just hangs, even if I don't actually use rembg.

[0x4E69676874466F78]

Doesn't seem to need special support. But model requires 320px tiles. When I try to set this size I get an error.

[ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: input.1 for the following indices
 index: 2 Got: 163 Expected: 320
 index: 3 Got: 163 Expected: 320
 Please fix either the inputs or the model.

Input values (partial):
• Image: RGB image 1080x1080
• Tile Size Target: 320

But in general, the model works:

[db0]

@theflyingzamboni you mean rembg in isolation? yes I use it a lot

[joeyballentine]

When I try to set this size I get an error.

Tile size target isn't a real tile size. it's a workaround I did to be able to manually set the splitting depth the upscale function uses, without having it be confusing to anyone who knows what a tile size is. It basically converts the tile size into whatever split depth would get you the largest size under the target size. I will be changing how this works soon to be a bit more verbose, and later modifying it to be an actual traditional tile size.

[0x4E69676874466F78]

This model requires the same patch as armorlab. in_nc -> 3 We clearly need support for such models, this is the second such one.

[0x4E69676874466F78]

I also did not understand what to do with the tiles. How to transfer a tile of the required size, which is required for these models? Need some sort of tile iterator?

[joeyballentine]

@0x4E69676874466F78 if you're referring to the new "number of tiles" dropdown, that's basically just the same thing as a tile size only it's splitting the image into equal-size tiles, the number of which is chosen with that dropdown. (the only reason it's like this is because I'm reusing my automatic splitting code, but with manually chosen splitting. I'll eventually change this, it just requires me to write a normal tile size algorithm and I'm procrastinating on that)

So, you don't need a tile iterator. I guess we currently don't support models with static width and height, unless you know ahead of time what the width and height it needs is, in which case you could manually crop. I'll have to do something to detect those static dimensions and basically just make it ignore any tile size setting passed in.

[0x4E69676874466F78]

@joeyballentine In theory, for static tile sizes, you need to complement the image size to a multiple of it. Example: 320 tile size. Image 1024х1024 add border to get the size 1280‬x1280‬ and autocrop after.

[theflyingzamboni]

@db0 I don't know if you got a notification when this issue was closed, but I got around to adding this to chaiNNer. It's not in a release version yet, but is available in chaiNNer nightly.

[db0]

Yes I did indeed see it thanks for letting me know. Unfortunately I'm very busy with the ai horde atm so I can't test it. But great job nonetheless!