usyd-fsalab fp6_llm issues - Githubissues

usyd-fsalab / fp6_llm

An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).

Apache License 2.0

170 stars 14 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

End-to-end performance for ViTs

#11 shivmgg opened 3 weeks ago
0
Does the repo provide a quantization kernel?

#10 yatorho closed 4 weeks ago
4
Update README.md

#9 tpoisonooo closed 2 months ago
0
Performance regression for large batch sizes

#8 gau-nernst closed 3 months ago
5
Does fp6 need a calibration dataset to tune with?

#7 leiwen83 closed 4 weeks ago
2
How to create FP16 quantization scales?

#6 mgoin opened 3 months ago
2
Can we get FP4?

#5 catid opened 4 months ago
1
Has the accuracy and performance been compared with awq？

#4 dingjingzhen opened 4 months ago
2
How to support V100(sm70)

#3 fishelegs closed 4 weeks ago
1
build Qwen/Qwen1.5-14B-Chat error

#2 yoke233 closed 4 weeks ago
1
Refine README.

#1 JamesTheZ closed 5 months ago
0