search

intel / xFasterTransformer

Apache License 2.0
322 stars 56 forks source link

[API] Add LLaMA decoder API. #386

Closed changqi1 closed 2 months ago

changqi1 commented 2 months ago

@pujiang2018 Do you have some idea to export layernorm or rms_norm param in the decoder API? void invokeLayerLLaMA(DataType dt, ActivationType at, NormType nt, ....)

pujiang2018 commented 2 months ago

@pujiang2018 Do you have some idea to export layernorm or rms_norm param in the decoder API? void invokeLayerLLaMA(DataType dt, ActivationType at, NormType nt, ....)

I think like this is OK.

changqi1 commented 2 months ago

@pujiang2018 @Duyi-Wang Done