Question in Evaluate

[muzhaohui]

Hello there! First of all, thank you for your outstanding work! I have a problem when reproducing your work.

I'm use the following command to evaluate. python evaluate.py --config configs/mm-distillnet.cfg --checkpoint trained_models/mm-distillnet.0.pth.tar

But get bad performance.Can you help me how to improve?

Thanks!

[sunwell1994]

same problem here. Do you use the given version of python, Pytorch, and Cuda? My version is a bit different, but it should not degrade the performance so much during inference.

[muzhaohui]

同样的问题在这里。您是否使用给定版本的 python、Pytorch 和 Cuda？我的版本有点不同，但它不应该在推理过程中降低性能。

I found that the cause of the problem was the thermal teacher, because there was basically no output during the test. And the provided model cannot be fully imported when importing. I feel that the author’s thermal teacher model may not be EfficientDet_D2 in the code.

15

[sunwell1994]

同样的问题在这里。您是否使用给定版本的 python、Pytorch 和 Cuda？我的版本有点不同，但它不应该在推理过程中降低性能。

I found that the cause of the problem was the thermal teacher, because there was basically no output during the test. And the provided model cannot be fully imported when importing. I feel that the author’s thermal teacher model may not be EfficientDet_D2 in the code.

15

Are teacher models called during inference? I thought that teacher models are used during training to generate labels and only the audio regression model will be used for inference.

[catherine-qian]

hi, can I know your dataset structure?

Because from the readme, it should contains 50 directories with “ We split the data using a 60 (train) /20 (validation)/20 (test) scheme, which can be found under the SPLITS directory”: http://multimodal-distill.cs.uni-freiburg.de/static/dist/MAVD_dataset/README.txt?

When I download the dataset, I only get 12 directories with the following structures: 432K drive_day_2020_03_18_15_42_58 53M drive_day_2020_04_14_14_33_17 314M drive_day_2020_04_14_15_10_13 173M drive_day_2020_04_14_15_20_14 202M drive_day_2020_04_14_16_16_27 862M drive_day_2020_04_21_15_48_20 488M drive_day_2020_04_21_15_58_22 2.6G drive_day_2020_05_21_20_55_36 5.6G drive_night_2020_05_21_21_05_38 7.1G drive_night_2020_05_21_22_22_50 11G drive_night_2020_05_31_22_00_00 104M drive_night_2020_05_31_22_23_23

[eyeris]

@muzhaohui May I know how did you make it runnable? I used the same method but end up getting this.

Also, I have the following issue as well. Tried to reach the authors in many ways but didn't have any reply. https://github.com/robot-learning-freiburg/MM-DistillNet/issues/18

[muzhaohui]

@muzhaohui May I know how did you make it runnable? I used the same method but end up getting this.

Also, I have the following issue as well. Tried to reach the authors in many ways but didn't have any reply. #18

1. Check if there is any problem with the path
2. If it doesn't work, delete the offending data

[liushibei]

Have you solved this problem? The performance of the model I trained is very poor

The performance of the model provided by article are also very poor.

[RST2detection]

Have you solved this problem? The performance of the model I trained is very poor

The performance of the model provided by article are also very poor.

May I ask if you have solved this problem? Whether I retrain or test using the best model given by the author, the results are nowhere near the accuracy reported in the article.