search

yoshitomo-matsubara / torchdistill

A coding-free framework built on PyTorch for reproducible deep learning studies. 🏆25 knowledge distillation methods presented at CVPR, ICLR, ECCV, NeurIPS, ICCV, etc are implemented so far. 🎁 Trained models, training logs and configurations are available for ensuring the reproducibiliy and benchmark.
https://yoshitomo-matsubara.net/torchdistill/
MIT License
1.37k stars 132 forks source link

[BUG] Problems in DistillationBox #471

Closed 1396066796 closed 4 months ago

1396066796 commented 4 months ago

Please use Discussions to ask questions.

Describe the bug A clear and concise description of what the bug is.

In DistillationBox, the class instantiates the AuxiliaryModelWrapper before placing the model in DataParallel. I am attaching a screenshot below.

To Reproduce Provide Whether or not you made any changes in Python code (if so, how you made the changes?) none lib file changed Project files srd.zip

Expected behavior A clear and concise description of what you expected to happen.

Environment (please complete the following information):

Additional context ef74033aaf1b388412b6734623947a4

754c85d6e5ad615a2a6eaa869a184fe

f6e33c006bd1bc47be3601bcee7c2ea

065d4f5039b1ff9dedbaa6daea58710

yoshitomo-matsubara commented 4 months ago

Hi @1396066796

Thank you for the issue. The AuxiliaryModelWrapper should be wrapped by wrapper: DataParallel, and it looks like the config is incorrect. I'm fixing it now

yoshitomo-matsubara commented 4 months ago

It is not a bug in code, but in the config file. I updated the sample config file in the above PR. For the official SRD config, I will work with @roymiles in a separate PR