search

PeterL1n / BackgroundMattingV2

Real-Time High-Resolution Background Matting
MIT License
6.81k stars 950 forks source link

Doubts about evaluation on different datasets #165

Closed caihaunqai closed 2 years ago

caihaunqai commented 2 years ago

Hello, thank you for your excellent work. We know that our model is trained in a order on multiple datasets. Do you use the same operation， which means training on multiple datasets and evaluating on one certain dataset, when comparing with other methods on a certain dataset? I wonder if other methods in quantitative evaluation in the paper's Table 1 also use multiple datasets for training？ Thank you~

PeterL1n commented 2 years ago

Evaluation is all done on the final model, which is trained on VideoMatte, PhotoMatte, and Distinctions.

caihaunqai commented 2 years ago

Evaluation is all done on the final model, which is trained on VideoMatte, PhotoMatte, and Distinctions.

Thank you for your reply. Are other methods in paper's Table 1 also trained on VideoMatte, PhotoMatte, and Distinctions? If so, do they also train in order or use their papers' training methods?

PeterL1n commented 2 years ago

We retrained BGM on the three datasets and called it BGM adapt. Other methods are not if I remembered it correctly.

caihaunqai commented 2 years ago

Owner

@PeterL1n PeterL1n commented 15分钟前

Thank you. Yes, I think other methods may be not trained in the same way, because other methods require trimap or other input, and VideoMatte, PhotoMatte do not provide trimaps.