Is there any segmentation mask alignment/registration when calculating the aux seg loss?

neu-vi / SportsSloMo

SportsSloMo: A New Benchmark and Baseline Models for Human-centric Video Frame Interpolation, CVPR 2024 (https://arxiv.org/abs/2308.16876)

https://neu-vi.github.io/SportsSlomo/

64 stars 3 forks source link

Is there any segmentation mask alignment/registration when calculating the aux seg loss? #4

Closed Juliazy closed 11 months ago

Juliazy commented 12 months ago

Hi! Thanks for sharing the code and data!

I have a question regarding the auxiliary segmentation loss. When you use the segmentation mask of I_t and I_t_hat to calculate the CE loss and dice loss, is there any segment alignment/ registration step to make the two segmentation masks aligned (i.e. same index for same object)? If so, how did you do that?

Thanks again!

JiabenChen commented 11 months ago

Hi, thanks for your interest! In the segmentation loss design, the two segmentation masks are aligned with the "category_id" information. I will release the code for preparing segmentation masks and training using auxiliary loss soon.