performance loss for int4 compare with AWQ? - Githubissues

pytorch-labs / gpt-fast

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

BSD 3-Clause "New" or "Revised" License

5.37k stars 488 forks source link

performance loss for int4 compare with AWQ? #10

Open lucasjinreal opened 7 months ago

lucasjinreal commented 7 months ago

performance loss for int4 compare with AWQ?