search

Hao840 / OFAKD

PyTorch code and checkpoints release for OFA-KD: https://arxiv.org/abs/2310.19444
81 stars 11 forks source link

FitNet Baseline for Non-CNN arch #24

Closed anithselva closed 4 months ago

anithselva commented 4 months ago

Hello,

How were the FitNet baseline produced for non-CNN architectures (i.e. Transformer Teacher, CNN Student)?

There is an assert in https://github.com/Hao840/OFAKD/blob/main/distillers/fitnet.py#L17 to prevent non-CNN student and teacher architectures

Looking forward for your response.

anithselva commented 4 months ago

Closing this as I found the relevant information in https://github.com/Hao840/OFAKD/issues/10