search

HobbitLong / RepDistiller

[ICLR 2020] Contrastive Representation Distillation (CRD), and benchmark of recent knowledge distillation methods
BSD 2-Clause "Simplified" License
2.11k stars 389 forks source link

code for ensemble distillation #17

Closed tonmoy-saikia closed 4 years ago

tonmoy-saikia commented 4 years ago

Nice work! I was wondering if you are planning to release code for ensemble distillation?

HobbitLong commented 4 years ago

You can simply sum up all L_{crd}(T_i; S), with i indicating the i-th teacher.