only failsafe mode avalueable for unknown reason

[Zuzia-Sweetheart]

Hello, i have trouble with my home server. I own a dell T7500 with 2 intel xeons and 84gb of ram, i also have nvidia tesla c2075

Current system is the newest Linux mint

Currently koboldcpp allows me to only use failsafe mode, which is underwhelming as it means my nvidia card is just there doing nothing

cuBLAS most likely won't work as its an old card, but i hoped that CLblast or Vulkan would, but neither do anything

When i select Vulkan, instead of detecting ,,Nvidia tesla C2075'' like it does when i select CuBLAS (CuBLAS detects my nvidia but it crashes when turned on), instead it shows ,,lvmpipe (LLVM 15.0.7 [there is also some stuff further but it cuts off]) and when i select CLblast it shows nothing at all

I really wish to use my gpu in any way, so any help would be deeply appreciated

[LostRuins]

Can you provide your system specs? What CPU do you have?

When you select Vulkan, are there any other options within the GPU selection dropdown?

[Zuzia-Sweetheart]

X5675 intel. two of them to be specific (same one's), when i select vulkan on ,,gpu id 1'' it shows what i said previously, but otherwise it shows nothing

[LostRuins]

Ah that's a very old CPU from 2011, it doesn't have AVX2 or AVX instructions. The GPU is also ancient, and won't support the necessary CUDA functionalities.

[Zuzia-Sweetheart]

ye i suspected that, i just hoped that i could use the gpu in some way, CLblast or anything really, could you please help me with that? I live in a poorer country so any newer server wasn't an option unfortunately

[LostRuins]

Hmm sorry to hear that, but even if you could use that GPU it's so old it's unlikely to get any decent speed.

Have you considered using our Colab instead? You can run the models off google's GPUs https://colab.research.google.com/github/LostRuins/koboldcpp/blob/concedo/colab.ipynb

[Zuzia-Sweetheart]

Due to reasons i cannot really share publicly i would much rather use the self hosted options, and the lack of decent speed from my GPU is not a problem, right now it goes purely off of my cpu so any prompt is processed in between 200-500 seconds, so any form of boost from a gpu, even a small one would be greatly appreciated

So if you could i would really appreciate you helping me utilize my gpu in anyway, i could pay in case it would be neccesary, but since i am in a poorer country i couldn't pay too much unfortunately

[Zuzia-Sweetheart]

if its something useful, everytime i use failsafe mode, in the terminal i get ,,[GFX1-]: Couldn't sanitize GL_RENDERER "Tesla C2075/PCIe/SSE2" [GFX1-]: Couldn't sanitize GL_RENDERER "Tesla C2075/PCIe/SSE2"''

[Zuzia-Sweetheart]

The GPU i also got second hand, it was dirt cheap because some guy wanted to get rid of some electronics, so swapping the gpu would most likely not be an option as this one was for 20 dollars (i use american currency for reference, but its not my currency)

[henk717]

If the CPU doesn't even support AVX we only support the CPU only fallback mode. But since you are on linux you could try compiling it yourself while leaving out LLAMA_PORTABLE=1 so it compiles natively for your CPU.

[Zuzia-Sweetheart]

That is very unfortunate...

[Zuzia-Sweetheart]

but ye i will try the compile thing, thanks

[Zuzia-Sweetheart]

are there any possible additional flags i should use while compiling for better performance?

[Zuzia-Sweetheart]

would Lenovo ThinkPad T540P I7-4810MQ be a good option for small models? I found it second hand for dirt cheap. Or any other older yet still AVX compatibile mashine you recommend?

[henk717]

AVX2 is strongly recommended for performance / compatibility