Open arylwen opened 3 months ago
0.2.27 loads the default nomic embedding model on the GPU with no issues. 0.2.29 fails to load it on the GPU. Loading on the CPU works well.
No version after 0.2.27 could load any models on the GPU.
System: Ubuntu 22.04.4 LTS (GNU/Linux 6.5.0-45-generic x86_64) 0 updates can be applied immediately.
GPU - nvidia-smi Fri Jul 26 12:08:30 2024 +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 535.183.01 Driver Version: 535.183.01 CUDA Version: 12.2 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 NVIDIA GeForce GTX 970 Off | 00000000:01:00.0 Off | N/A | | 35% 31C P8 13W / 170W | 59MiB / 4096MiB | 0% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+ | 1 NVIDIA GeForce GTX 970 Off | 00000000:02:00.0 Off | N/A | | 35% 29C P8 13W / 170W | 6MiB / 4096MiB | 0% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+
0.2.27 loads the default nomic embedding model on the GPU with no issues. 0.2.29 fails to load it on the GPU. Loading on the CPU works well.
No version after 0.2.27 could load any models on the GPU.
System: Ubuntu 22.04.4 LTS (GNU/Linux 6.5.0-45-generic x86_64) 0 updates can be applied immediately.
GPU - nvidia-smi Fri Jul 26 12:08:30 2024 +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 535.183.01 Driver Version: 535.183.01 CUDA Version: 12.2 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 NVIDIA GeForce GTX 970 Off | 00000000:01:00.0 Off | N/A | | 35% 31C P8 13W / 170W | 59MiB / 4096MiB | 0% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+ | 1 NVIDIA GeForce GTX 970 Off | 00000000:02:00.0 Off | N/A | | 35% 29C P8 13W / 170W | 6MiB / 4096MiB | 0% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+