Open yli147 opened 1 month ago
Any hint ? Thanks
looks like the memory of the gpu it is not enough for what the program needs
looks like the memory of the gpu it is not enough for what the program needs
No memory issue if the same gpu card on x86 host. So this is somehow strange
GPU内存不够了吧,你跑x86 host用的内存和GPU的内存不是同一块。host memory够用,device memory不够用
I was reverse engineering the code in this past 1 day and 12 hours(bruh), i encontred a ram alocation in a gpu code. This is a problem?
in the log is saying: ggml_vulkan: Device memory allocation of size 1610612736 failed. ggml_vulkan: vk::Device::allocateMemory: ErrorOutOfDeviceMemory
maybe, the computer not have enough RAM for run the program
Thanks @Lilicogamer13 @warren-lei after changing to a smaller model (OpenELM), finally it can work now :)
What happened?
I have an Intel ARC750 graphic card. The same Phi-3-mini-4k-instruct-fp16.gguf can be run on x86 host with vulkan backend successfully, but it failed on RISC-V host
Name and Version
./llama-cli --version version: 3372 (a977c115) built with cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 for riscv64-linux-gnu
What operating system are you seeing the problem on?
Linux
Relevant log output