search

THUDM / SwissArmyTransformer

SwissArmyTransformer is a flexible and powerful library to develop your own Transformer variants.
https://THUDM.github.io/SwissArmyTransformer
Apache License 2.0
951 stars 90 forks source link

fix lost bias when quantize from pre-trained model parameters #134

Closed jimmieliu closed 12 months ago

jimmieliu commented 12 months ago

Previously, if you quantize a model with pre-trained parameter, the bias parameters in Linear will be lost, and you will get a all-zero bias. 修复quantize时候丢掉模型Linear的bias的问题。

jimmieliu commented 12 months ago

updated with a new PR