ZhengPeng7 / BiRefNet

[CAAI AIR'24] Bilateral Reference for High-Resolution Dichotomous Image Segmentation
https://www.birefnet.top
MIT License
1.21k stars 95 forks source link

So many models #58

Closed rishabh063 closed 3 weeks ago

rishabh063 commented 2 months ago

Hey @ZhengPeng7 I have been busy not been able to keep up with your progress

I see many models on the drive ( stuff )

which is the best ones these days ?

Are you training even better one ?

Any comparison with closed source model now

ZhengPeng7 commented 2 months ago

Hi, @rishabh063 , you can see the GitHub release and my Hugging Face models for those for practical use. Practically, you can directly use the one for general use, which is always maintained and updated on https://huggingface.co/ZhengPeng7/BiRefNet.

ZhengPeng7 commented 2 months ago

I didn't have time to do the comparison with closed models, but some others did it. However, there are only cases not a comprehensive one.

cyy-1234 commented 2 months ago

Hi,Peng,I would like to ask among the many versions, which one is better at segmenting real pictures with more obvious subjects. I look forward to your reply. image

ZhengPeng7 commented 2 months ago

BiRefNet-general-epoch_244.pth should be the best one for you. And, of course, you can go to HF demo to have a try online with different weights on some of your samples.

Hi,Peng,I would like to ask among the many versions, which one is better at segmenting real pictures with more obvious subjects. I look forward to your reply. image

cyy-1234 commented 2 months ago

Hi,I used your excellent model(BiRefNet-general-epoch_244.pth),First of all, I think this model is very excellent,And believe that the future will be even better. and found that there are some segmentation problems in actual pictures,

  1. Mirror problem image image image

  2. Transparency problem image image Some are transparent and very good!!! img_v3_02e0_d673e77f-b936-4944-afb7-766c55f4266g img_v3_02e0_87db1a0f-a95e-4093-a6dc-2eb98a37dd6g

  3. Hair problem (this is difficult) image 4、At the same time, sometimes I feel like there are white edges(Appears mostly on white background) image 5、reflection problem image

ZhengPeng7 commented 2 months ago

Thanks @cyy-1234 ! These bad cases are really valuable, and I will think about their problems. And your classification of these problems does help, I appreciate it :)

cyy-1234 commented 2 months ago

Hi,Peng,I have tried many times and found that when the model is mounted on a plate, it is easy to miss the points on the plate. This phenomenon is very high. I don’t know if it is a problem. image image

ZhengPeng7 commented 2 months ago

Hi, cyy, that's caused by that the subject in the image seems to be the fruit/food, instead of the plate, which is recognized as the background. Since it's automatic subject extraction, this phenomenon is hard to avoid (it's correct).

Even so, I think you may be able to obtain the results you want through some tricks. For example, you can try with the box-guided segmentation colab -- add some background surroundings to make the plate and things on it look like a whole, then give a box surround the plate.

ZhengPeng7 commented 2 months ago

I tried it by myself, it's currently impossible to include things that are not subject. It may need prompts (points/box) as what's used in SAM.

ZhengPeng7 commented 2 months ago

Hi,Peng,I have tried many times and found that when the model is mounted on a plate, it is easy to miss the points on the plate. This phenomenon is very high. I don’t know if it is a problem. image image

Hi, cyy, I surprisingly found that model trained on DIS5K only seems better on these cases (that's caused by how the labels are made in the datasets), you can directly use this model: https://huggingface.co/ZhengPeng7/BiRefNet-DIS5K-TR_TEs. Here are the results of your cases above (I downloaded your screenshot and did a crop).

截屏2024-08-23 22 42 24 截屏2024-08-23 22 45 32

cyy-1234 commented 2 months ago

wow,Awesome, I will try the model you recommended. @ZhengPeng7