Can not reproduce table3 with the provided checkpoint.

LHY-HongyangLi commented 9 months ago

Hi @nikitakaraevv , I tried to reproduce the performance of cotrackerv2 using the checkpoint you have provided on DAVIS-First in "glob. 5×5" mode. But I got the following results, which are not the same as you have posted in table3, is there anything wrong?

evaluate_result {'occlusion_accuracy': 0.8830991955685764, 'pts_within_1': 0.41818242347516404, 'jaccard_1': 0.27439414118807914, 'pts_within_2': 0.6585306168489217, 'jaccard_2': 0.4944025070918661, 'pts_within_4': 0.8213521434366008, 'jaccard_4': 0.67601993042582, 'pts_within_8': 0.900469386477676, 'jaccard_8': 0.7759953203865924, 'pts_within_16': 0.9396396614828214, 'jaccard_16': 0.8167342923773883, 'average_jaccard': 0.607509238293949, 'average_pts_within_thresh': 0.7476348463442369}

nikitakaraevv commented 9 months ago

Hi @LHY-HongyangLi, the results may vary depending on the version of different libraries used in the code, particularly PyTorch. What's your PyTorch version?

LHY-HongyangLi commented 8 months ago

Sorry for my late reply. The version of the pytorch-related packages in our environment are as follows

torch                     1.13.0+cu117
torchvision               0.14.0+cu117

I'll close this issue since the lower results are not caused by CoTracker.

facebookresearch / co-tracker

Can not reproduce table3 with the provided checkpoint. #68