pinokiocomputer / llamanet

Replace OpenAI with Llama.cpp Automagically.
https://llamanet.netlify.app
MIT License
277 stars 14 forks source link

spawn command fails in llamacpp.js with Error: spawn ./server ENOENT #5

Closed AlextheYounga closed 2 hours ago

AlextheYounga commented 2 hours ago

Steps to reproduce:

node bin/index.js

Output:

█ llamanet running at http://localhost:42424

[QUICKSTART] Try opening a new terminal and run the following command.

  curl --request POST \
      --url http://127.0.0.1:42424/v1/chat/completions \
      --header "Content-Type: application/json" \
      --data '{
        "model": "https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-q4.gguf",
        "messages": [
          { "role": "system", "content": "You are a helpful assistant." },
          { "role": "user", "content": "Do aliens exist?" }
        ]
      }'

In a new terminal, I pasted the curl command:

  curl --request POST \
      --url http://127.0.0.1:42424/v1/chat/completions \
      --header "Content-Type: application/json" \
      --data '{
        "model": "https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-q4.gguf",
        "messages": [
          { "role": "system", "content": "You are a helpful assistant." },
          { "role": "user", "content": "Do aliens exist?" }
        ]
      }'

./server child process fails to spawn in line 44 of llamacpp.js:

https://github.com/pinokiocomputer/llamanet/blob/7e288ccba1721014102d9e1886184f3ecfceeca4/llamacpp.js#L44

With the following output (debug mode on):

./server -c 2048 --embeddings -m /Users/alexyounger/llamanet/models/huggingface/microsoft/Phi-3-mini-4k-instruct-gguf/Phi-3-mini-4k-instruct-q4.gguf --port 8000
node:events:492
      throw er; // Unhandled 'error' event
      ^

Error: spawn ./server ENOENT
    at ChildProcess._handle.onexit (node:internal/child_process:286:19)
    at onErrorNT (node:internal/child_process:484:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
Emitted 'error' event on ChildProcess instance at:
    at ChildProcess._handle.onexit (node:internal/child_process:292:12)
    at onErrorNT (node:internal/child_process:484:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn ./server',
  path: './server',
  spawnargs: [
    '-c',
    '2048',
    '--embeddings',
    '-m',
    '/Users/alexyounger/llamanet/models/huggingface/microsoft/Phi-3-mini-4k-instruct-gguf/Phi-3-mini-4k-instruct-q4.gguf',
    '--port',
    '8000'
  ]
}

Node.js v20.10.0

Tested on Mac and Linux

AlextheYounga commented 2 hours ago

Didn't realize this was a duplicate of #2. Closing