search

go-skynet / go-llama.cpp

LLama.cpp golang bindings
MIT License
626 stars 78 forks source link

GGML_ASSERT: /build/go-llama/llama.cpp/ggml-cuda.cu:6670: src0->type == GGML_TYPE_F16 #261

Closed yourtiger closed 9 months ago

yourtiger commented 9 months ago

My operating system is Centos7 image I download and use quay.io/go-skynet/local-ai:master-cublas-cuda12 So, I installed CDUA on the operating system that matches this image image I user ggml-model-q4_0.gguf（Llama2-13B-chat） yaml file in the model folder image I use Debug mode to run the image of LocalAI image An error occurred when I used Postman to send inference image The error message for LocalAI is as follows image

I can reason normally by running llama.cpp separately in the container But there was an error running the test under go lama image ...... image @deadprogram @mauromorales @jrc2139 @soleblaze

yourtiger commented 9 months ago

Repeated submission