Closed xiezipeng-ML closed 1 year ago
[x] use rms_norm
[x] use fused_self_attention
[x] use fused_bias_add_scale_mask_softmax_dropout
[x] use fused_fast_gelu_mul
[x] test training
[x] modify inference
[x] test inference
[x] use rms_norm
[x] use fused_self_attention
[x] use fused_bias_add_scale_mask_softmax_dropout
[x] use fused_fast_gelu_mul
[x] test training
[x] modify inference
[x] test inference