search

bytedance / lightseq

LightSeq: A High Performance Library for Sequence Processing and Generation
Other
3.22k stars 329 forks source link

请问对于LSTM的量化，有用cuBLASLt库实现的方案吗？ #449

Closed piaohe20221128 closed 1 year ago

piaohe20221128 commented 1 year ago

在代码中看到对于LSTM用的是传统的量化方法，即用的float的GEMM计算的，请问有没有用int8 GEMM计算的方案呢？非常感谢！！！

neopro12 commented 1 year ago

Currently LSTM models are not supported. For Transformer models, we use cuBLASLt for int8 GEMM.

piaohe20221128 commented 1 year ago

thank you