search

open-mmlab / mmrazor

OpenMMLab Model Compression Toolbox and Benchmark.
https://mmrazor.readthedocs.io/en/latest/
Apache License 2.0
1.45k stars 228 forks source link

How to distill layers with different number of filters? #589

Open 294486709 opened 11 months ago

294486709 commented 11 months ago

In mmrazor, distill examples with CWD works on the condition that teacher/student layer has the same number of feature channels. Is there a way to apply CWD on layers with different number of filters?

SweeneyLi commented 2 months ago

I have the same problem. The other solution I see is to add the 1*1 conv to change the channel of the student model manually.