How to generate Trimap?

[ofirkris]

Hi, Thanks for the implementation, results are really good, and look similar to the paper indeed.

Any suggestion on how to generate Trimap masks? Also, is there a plan to release training code?

[poppinace]

At the inference stage, trimaps are provided by the Adobe Image Matting dataset. At the training stage, trimaps can be generated on-the-fly using provided GT alpha mattes. I use the Euclidean distance transform to dilate the unknown region. It has a similar effect compared to dilation, but it is more controllable. In fact, I was more than happy to release the training code, while my boss seemed to have another plan and told me not to release at present. ╮(╯▽╰)╭

[ofirkris]

From what I see you cannot get matting without a provided Trimap\Segmentation. I removed the "Trimap" folder, and ran the demo script and got the following error:

FileNotFoundError: [Errno 2] No such file or directory: './examples/trimaps/beach-747750_1280_2.png'

I'm trying to find a solution for getting both matting and Trimap for my own images, is this possible? or do I need to combine an additional solution as Detectron or Mask_RCNN to get the Trimap for custom images?

[poppinace]

Yes, our model receives 4-channel input. Using the trimap is a common practice for natural image matting, as the model does not know what is the object of interest. In general, a trimap needed as it can tell such a prior to the model.

However, if your applicational scenario is specific, such as portrait matting, it is possible to generate a trimap automatically, e.g., with a segmentation model. Alternatively, you may seek a saliency network to generate trimaps if the foreground is salient against the background.

[wrrJasmine]

@poppinace Thanks for your great work. Would you consider to release the code about how to generate trimaps?

[poppinace]

@wrrJasmine Please take a look at the following pieces of code how I generate the trimaps during training:

def generate_trimap(self, alpha):
    fg = np.array(np.equal(alpha, 255).astype(np.float32))
    unknown = np.array(np.not_equal(alpha, 0).astype(np.float32))
    unknown = unknown - fg
    unknown = morphology.distance_transform_edt(unknown==0) <= np.random.randint(1, 20)
    trimap = fg * 255
    trimap[unknown] = 128
    return trimap.astype(np.uint8)

Trimaps affect the training a lot! Always check what you generate.

Best

[wrrJasmine]

@poppinace thank you a lot