Alpha matting

[Mathnerd314]

From what I can tell you are doing the basic "naive" cutout compositing (here). There is some code in rembg called "alpha matting" which is a little more involved (same model, just different post-processing): https://github.com/danielgatis/rembg/blob/main/rembg/bg.py#L34-L100. The projects are both MIT licensed so there is no license issue.

Comparison (try viewing at 1:1 pixel ratio in a new tab and setting the CSS background-color property to black/white/magenta with devtools):

Naive	Alpha Matting

Last is my image (I am making an alcohol tracker, the plan is that users will take pictures of cans and bottles). To my eyes all of the cutouts are just a little better with the alpha matting, less of a "halo". For example the junk on the left side of the bear is less prominent. I haven't timed the additional processing required but I think it is minimal compared to running the model, so even if the effect is unnoticeable for most use cases I think it is worth porting.

Here is how I generated the images in Colab:

!pip install rembg
from rembg import new_session, remove
from PIL import Image
from pathlib import Path

session = new_session("u2netp")
for file in Path('/content').glob('*.jpg'):
    input_path = str(file)
    output_path = str(file.parent / (file.stem + ".naive.png"))
    output_path2 = str(file.parent / (file.stem + ".alpha.png"))

    input = Image.open(input_path)
    output = remove(input, session=session)
    output.save(output_path)
    input2 = Image.open(input_path)
    output2 = remove(input2, session=session,
                     bgcolor=(0, 0, 0, 0),
                     alpha_matting=True, alpha_matting_foreground_threshold=270,alpha_matting_background_threshold=20, alpha_matting_erode_size=11)
    output2.save(output_path2)

[Mathnerd314]

Looking at it a little more it uses the pymatting library, closed-form method. But it is yet again MIT license.

[erenalpaslan]

Thank you for your support i will check it

[DevTrinh]

How do I improve image quality when removing background?

I am using a simple code like this project instructs private fun getMaskedImage(input: Bitmap, mask: Bitmap): Bitmap { val result = Bitmap.createBitmap(mask.width, mask.height, Bitmap.Config.ARGB_8888) val mCanvas = Canvas(result) maskPaint.alpha = 240 mCanvas.drawBitmap(input, 0f, 0f, null) mCanvas.drawBitmap(mask, 0f, 0f, maskPaint) return result }

So how to add attributes like alpha_matting, alpha_matting_foreground_threshold, alpha_matting_background_threshold,... ?

[Mathnerd314]

It is not implemented in this library (removebg), someone would have to implement it by porting it from Python (rembg/pymatting). I will probably do it myself eventually if nobody else does, but it is not currently a priority for me (the improvements for my use case are minimal).

If you want to do it, I have already linked the relevant rembg code, and the relevant pymatting code is estimate_alpha_cf (which uses cf_laplacian) and estimate_foreground_ml. It is maybe 500 lines of matrix math in Python. The biggest question is what matrix library to use. This project already uses pytorch's Java bindings so probably the cleanest way would be to implement it with those, but I am not that familiar with pytorch.