[Feature Request] Add support for SM75

punica-ai / punica

Serving multiple LoRA finetuned LLM as one

https://arxiv.org/abs/2310.18547

Apache License 2.0

883 stars 40 forks source link

Open sleepwalker2017 opened 3 months ago

sleepwalker2017 commented 3 months ago

Any plans to add support for SM75 like V100 GPUs? Thank you!

yanxiyue commented 3 months ago

+1. I found that the third-party library flashinfer has already supported sm75. I'm wondering if there is a possibility of support for it now.

WangxuP commented 3 months ago

sleepwalker2017 commented 3 months ago

+1. I found that the third-party library flashinfer has already supported sm75. I'm wondering if there is a possibility of support for it now.

The code runs ok on SM version >= 7.0. But the performance may be much lower, because the code need the new feature of ampere arch.

oushu1zhangxiangxuan1 commented 2 months ago