search

xian-sh / MRNet

MIT License
3 stars 0 forks source link

Failed to reproduce your results #1

Open shzirui opened 3 months ago

shzirui commented 3 months ago

Thanks for your code, but I failed to reproduce your results on Charades and TACoS datasets with your provided data. Is there any change need to do in your original config?

xian-sh commented 3 months ago

Thank you for your attention. I will check and update, and provide feedback to you.

xian-sh commented 3 months ago

Thanks for your code, but I failed to reproduce your results on Charades and TACoS datasets with your provided data. Is there any change need to do in your original config?

The configuration file and weights have been updated for the ActivityNet and TACoS datasets, for the Charades dataset please wait for me to check the previous code before I provide it to you! This will take some time.

shzirui commented 3 months ago

Thanks for your reply! Following your new instructions, I successfully reproduce the results on the ActivityNet dataset. But I still get low results on the tacos dataset by changing the ret_model.py like: +------------+------------+------------+------------+------------+------------+-------+ | R@1,IoU0.1 | R@1,IoU0.3 | R@1,IoU0.5 | R@5,IoU0.1 | R@5,IoU0.3 | R@5,IoU0.5 | mIoU | +------------+------------+------------+------------+------------+------------+-------+ | 35.72 | 24.24 | 14.82 | 71.56 | 52.49 | 36.32 | 16.82 | +------------+------------+------------+------------+------------+------------+-------+ Is there any more modification should I do?

xian-sh commented 3 months ago

Thanks for your reply! Following your new instructions, I successfully reproduce the results on the ActivityNet dataset. But I still get low results on the tacos dataset by changing the ret_model.py like: +------------+------------+------------+------------+------------+------------+-------+ | R@1,IoU0.1 | R@1,IoU0.3 | R@1,IoU0.5 | R@5,IoU0.1 | R@5,IoU0.3 | R@5,IoU0.5 | mIoU | +------------+------------+------------+------------+------------+------------+-------+ | 35.72 | 24.24 | 14.82 | 71.56 | 52.49 | 36.32 | 16.82 | +------------+------------+------------+------------+------------+------------+-------+ Is there any more modification should I do?

I have updated the file "ret/data/datasets/tacos.py" , you can update it and try again.

hao-yang-bot commented 2 months ago

Thanks for your code, but I have two question: 1、Using your weight, I failed to reproduce your results on TACoS datasets. That is my result by using your weight. +------------+------------+------------+------------+------------+------------+-------+ | R@1,IoU0.1 | R@1,IoU0.3 | R@1,IoU0.5 | R@5,IoU0.1 | R@5,IoU0.3 | R@5,IoU0.5 | mIoU | +------------+------------+------------+------------+------------+------------+-------+ | 71.51 | 55.96 | 40.66 | 89.33 | 78.51 | 65.98 | 39.08 | +------------+------------+------------+------------+------------+------------+-------+ 2、Do you work in an environment of reproducibility? By running your code I find your code is not reproducible. I ran your code twice with different results.

xian-sh commented 2 months ago

Thanks for your code, but I have two question: 1、Using your weight, I failed to reproduce your results on TACoS datasets. That is my result by using your weight. +------------+------------+------------+------------+------------+------------+-------+ | R@1,IoU0.1 | R@1,IoU0.3 | R@1,IoU0.5 | R@5,IoU0.1 | R@5,IoU0.3 | R@5,IoU0.5 | mIoU | +------------+------------+------------+------------+------------+------------+-------+ | 71.51 | 55.96 | 40.66 | 89.33 | 78.51 | 65.98 | 39.08 | +------------+------------+------------+------------+------------+------------+-------+ 2、Do you work in an environment of reproducibility? By running your code I find your code is not reproducible. I ran your code twice with different results.

The annotation files and many parts of the implementations are borrowed from MMN. You can try this code.