I tried to reproduce the speedup result on 1 A100 80GB with Llama 2 13b chat. But i failed to reproduce the given speedup result on llama2 13b chat (x3.03).
The results are as below.
speed 102.61022168131326
speed0 39.460857776166044
ratio 2.600303882478927
It would be thankful if you can provide your result or any advice.
Liyuhui-12
commented
2 months ago
Thanks for sharing your great works!
I tried to reproduce the speedup result on 1 A100 80GB with Llama 2 13b chat. But i failed to reproduce the given speedup result on llama2 13b chat (x3.03).
The results are as below. speed 102.61022168131326 speed0 39.460857776166044 ratio 2.600303882478927
It would be thankful if you can provide your result or any advice.