search

aifoundry-org / MHAQ

Customizable and automated environment for quantization algorithms.
Apache License 2.0
0 stars 0 forks source link

Initial distillation implementation #17

Closed janchk closed 1 month ago

janchk commented 2 months ago

Use inference from FP checkpoint as teacher with L2 norm to implement knowledge distillation.

janchk commented 2 months ago

L2_NORM(fp_out, quant_out) -> loss funct