search

SJTU-IPADS / PowerInfer

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs
MIT License
7.9k stars 406 forks source link

Some question about Fig4. #213

Open rhmaaa opened 2 months ago

rhmaaa commented 2 months ago

I want to reproduce the results of fig4 in your paper, but I have encountered many problems. Can you provide some ideas or codes?

rhmaaa commented 2 months ago

@ZeyuMi @eltociear @MatthewCroughan @hodlen

MatthewCroughan commented 2 months ago

There's a lot of missing information that makes reproducibility hard, and the code for the newer PowerInfer-2 is not yet open, I've found.

rhmaaa commented 2 months ago

There's a lot of missing information that makes reproducibility hard, and the code for the newer PowerInfer-2 is not yet open, I've found.

Thank you! I was just wondering how you got the data in Figure 4 and how to modify dejavu? image

rhmaaa commented 2 months ago

Use cupy to load model ?

rhmaaa commented 2 months ago

@MatthewCroughan