Any suggestions on how to improve segmenting a fluffy animal?

[ynie]

Thanks for the model again. I tried it with my dog and the masking is not great for his fur.

Any suggestions? Thank you very much!

[ZhengPeng7]

What a cute dog 🐶!

Reason: Honestly, it should be hard to segment due to the lack of this data in common segmentation datasets (e.g., SOD datasets, DIS datasets). However, this kind of data does appear a lot in image matting datasets.

Solution: I cannot download your uploaded image as it's shown as "private xxx" when clicked. But I made a screenshot of it (image quality is lower) and tried it with BiRefNet for portrait segmentation, of which the result is shown below. You can try that model on https://fal.ai/models/fal-ai/birefnet/playground (choose the model as portrait), where I collaborated with nice guys from that company to build the BiRefNet online demo.

The BiRefNet-portrait was not trained very seriously, and I plan to re-train it better in the next three days. I'll release the new model and update my online demo on HuggingFace with it. Even though, the predicted result of the low-quality screenshot is still better than the previous one.

Once I finish the training, I'll tell you here if you want.

[ynie]

Awesome! It does look better, and I'm looking forward for the weight. Few more questions:

1) Will we be able to host the weight ourselves? 2) Can we use the existing ComfyUI plugin for the portrait model? Is it just a weight swap?

Thank you!

[ZhengPeng7]

Sure, all the content of my project is under the MIT license. It's fully open-source and even commercially available.

Regarding ComfyUI, there are several third-party implementations of adding BiRefNet as a node. However, they haven't been updated for a long time, and so many things have been updated. Therefore, they may cause some inconsistency problems. They did it with interest and enthusiasm and made it open-source, so I cannot urge them to update with my official latest versions. Based on that, I will establish a ComfyUI node for BiRefNet by myself. It would take some time since I'm not familiar with ComfyUI, but since people like you do have this need, I'll surely do it.

[ynie]

Awesome! ComfyUI node will be so useful and the community will be benefit from it. Thank you!

[ynie]

Another great example here:

General Use Heavy:

Photoroom on iOS:

It would be great if the model can also trim the white border like photoroom as well. Thank you!

[ZhengPeng7]

Thanks for your excellent suggestion! I'll check if this is a common problem in the predicted results of BiRefNet. If so, I'll try to think about how to solve it through training or some post-processing.

But I also feel pleased that BiRefNet performs better on some other points 😄

[ynie]

Omg yes, I didn't even catch it. Great job! I'm working on a large batch of photos that need to remove the background and I will post the results in this issue:

https://github.com/ZhengPeng7/BiRefNet/issues/44