VILA - a multi-image visual language model with training, inference and evaluation recipe, deployable from cloud to edge (Jetson Orin and laptops)
972
stars
68
forks
source link
Add support for GPUs with compute capability lower than 8.0 for awq/kernels installation #45
Closed
rahulthakur319 closed 1 week ago
I tried to install and run the project on a machine with an NVIDIA Tesla T4 GPU, which has a compute capability of 7.5 (SM 75).
Environment Ubuntu 22.04 with CUDA 12.1
I followed the steps as mentioned here https://github.com/mit-han-lab/llm-awq/tree/main?tab=readme-ov-file#install & encountered the following error during the third step installation process:
Following error was reported
Root Cause: Feature '.m16n8k16' requires .target sm_80 or higher
Is there a configuration flag or workaround to support GPUs with capacity below 8.0