search

yoshitomo-matsubara / torchdistill

A coding-free framework built on PyTorch for reproducible deep learning studies. 🏆25 knowledge distillation methods presented at CVPR, ICLR, ECCV, NeurIPS, ICCV, etc are implemented so far. 🎁 Trained models, training logs and configurations are available for ensuring the reproducibiliy and benchmark.
https://yoshitomo-matsubara.net/torchdistill/
MIT License
1.37k stars 132 forks source link

Incorrect conditional judgement #467

Closed 1396066796 closed 4 months ago

1396066796 commented 4 months ago

https://github.com/yoshitomo-matsubara/torchdistill/blob/1fe3088e64fd39f23bc031720939a6dcbfec08a6/torchdistill/core/distillation.py#L418

When using distributed training (e.g. using DataParallel), the code uses isinstance to determine if the parent class of a strength is an AuxiliaryModelWrapper or not, which is an error, because the instance object that isinstance is judging is always DataParallel or DistDataParallel, which means that the judgement won't take effect.

yoshitomo-matsubara commented 4 months ago

Hi @1396066796

Is it your guess or did you confirm that training that uses AuxiliaryModelWrapper does not detect it? When using AuxiliaryModelWrapper, you should use DataParallel or DistributedDataParallel inside the wrapper, thus the current implementation should be correct.

Feel free to reopen this issue if you confirm it is not the case.

yoshitomo-matsubara commented 4 months ago

P.S. Please follow the bug report template when you open an issue