Closed chris-hatton closed 1 year ago
got the same error when use the curl command, deployed localai in k8s cluster
curl http://xxxx/v1/chat/completions -H "Content-Type: application/json" -d '{ "model": "koala-7B-4bit-128g.bin", "messages": [{"role": "user", "content": "Say this is a test!"}], "temperature": 0.7 }'
localai-local-ai.log
Closing, dup of #88.
I have LocalAI hosted in a docker container. Calling
models
endpoint provides expected output:But providing the example prompt - at either of those models - yields an opaque looking error:
Is there anything more I can try to help diagnose the reason?
I am running this on an HP z800 Workstation which is a fairly old machine using Dual Xeon X5570 CPU's. These don't have the AVX instruction set, in case that's a hard requirement, with
proc/cpuinfo
being:Standalone llama.cpp works, albeit slowly.