Open yzh119 opened 6 months ago
Thanks @yzh119! Currently vLLM support the following head sizes, all multiply of 16: 64/80/96/112/128/256
head size 112 is need +1 ~ thanks~
head size 112 is need +1 ~ thanks~
@yzh119
@yzh119 "Hello, I would like to ask when will 'head dim 64' be supported?"
Support more head dimensions to 64/128/256