search

bytedance / lightseq

LightSeq: A High Performance Library for Sequence Processing and Generation
Other
3.22k stars 329 forks source link

模型转换问题：encode_output_project_kernel_kv #318

Open 13354236170 opened 2 years ago

13354236170 commented 2 years ago
  1. 请问模型转换的时候 encode_output_project_kernel_kv 这个参数是来自于什么操作呢？我是使用的torch自带的decoder 构建的模型，没有找到对应的参数转换到这个参数上呢，我看大小要求hidden_sizehidden_size2 *dec_layer_num 。
  2. 请问构建TransformerDecoder 推理引擎的时候有什么办法跳过这个encode_output_project_kernel_kv 操作吗？
Taka152 commented 2 years ago

It's from kv projection in decoder cross attention which should not be skipped.