search

GaiaNet-AI / gaianet-node

Install and run your own AI agent service
https://www.gaianet.ai/
GNU General Public License v3.0
4.81k stars 210 forks source link

/bin/gaianet: line 569: 56621 Illegal instruction (core dumped) when starting LlamaEdge API Server in GaiaNet #111

Open billionaire opened 2 months ago

billionaire commented 2 months ago

Steps to Reproduce:

  1. Run curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash
  2. Run source /root/.bashrc
  3. Run /root/gaianet/bin/gaianet init
  4. Run /root/gaianet/bin/gaianet start

Expected Result: The LlamaEdge API Server should start without issues.

Actual Result: Error: Illegal instruction (core dumped)

Logs and System Information:

Output from `gaianet start` command:

root@220361:~# gaianet start
[+] Checking the config.json file ...

You already have a private key.
[+] Starting Qdrant instance ...
[+] Starting LlamaEdge API Server ...

    Run the following command to start the LlamaEdge API Server:

wasmedge --dir .:./dashboard --env NODE_VERSION=0.2.2 --nn-preload default:GGML:AUTO:Phi-3-mini-4k-instruct-Q5_K_M.gguf --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf rag-api-server.wasm --model-name Phi-3-mini-4k-instruct,Nomic-embed-text-v1.5 --ctx-size 4096,512 --batch-size 16,512 --prompt-template phi-3-chat,embedding --rag-policy system-message --qdrant-collection-name default --qdrant-limit 1 --qdrant-score-threshold 0.5 --web-ui ./ --socket-addr 0.0.0.0:8080 --rag-prompt "You are a tour guide in Paris, France. Use information in the following context to directly answer the question from a Paris visitor.\n----------------\n" 

/root/gaianet/bin/gaianet: line 569: 56621 Illegal instruction     (core dumped) nohup "${cmd[@]}" > $log_dir/start-llamaedge.log 2>&1
    LlamaEdge API Server started with pid: 56621

Output from /root/gaianet/bin/gaianet --version:

GaiaNet CLI Tool v0.2.2

Output from free -m:

               total        used        free      shared  buff/cache   available
Mem:            9940        1082         221           2        8636        8540
Swap:              0           0           0

Output from uname -a:

Linux 220361.xorek.cloud 5.15.0-75-generic #82-Ubuntu SMP Tue Jun 6 23:10:23 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Output from lscpu:

Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         48 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  6
  On-line CPU(s) list:   0-5

Output from ulimit -a:

real-time non-blocking time  (microseconds, -R) unlimited
core file size              (blocks, -c) 0
data seg size               (kbytes, -d) unlimited
scheduling priority                 (-e) 0
file size                   (blocks, -f) unlimited
pending signals                     (-i) 39309
max locked memory           (kbytes, -l) 1272444
max memory size             (kbytes, -m) unlimited
open files                          (-n) 1024
pipe size                (512 bytes, -p) 8
POSIX message queues         (bytes, -q) 819200
real-time priority                  (-r) 0
stack size                  (kbytes, -s) 8192
cpu time                   (seconds, -t) unlimited
max user processes                  (-u) 39309
virtual memory              (kbytes, -v) unlimited
file locks                          (-x) unlimited

Please let me know if you need any further information. start-llamaedge.log

rajkc3196 commented 2 months ago

Same error.. can't resolve till now.. if you get any solution. please post it here... Even the Gaianet team can't solve it...

billionaire commented 1 month ago

@juntao @alabulei1 @CodeAunt any news?

juntao commented 1 month ago

It is most likely that your CPU does not support the modern AVX instructions.

Can you try to update WasmEdge to an no-avx version?

curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install_v2.sh | bash -s -- -v 0.13.5 --noavx

You can try gaianet start again after that. Thanks.

amirsohail208 commented 1 month ago

image image

billionaire commented 1 month ago

It is most likely that your CPU does not support the modern AVX instructions.

Can you try to update WasmEdge to an no-avx version?

curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install_v2.sh | bash -s -- -v 0.13.5 --noavx

You can try gaianet start again after that. Thanks.

still no changes. /root/gaianet/bin/gaianet: line 584: 690713 Illegal instruction (core dumped) nohup "${cmd[@]}" > $log_dir/start-llamaedge.log 2>&1

# cat ~/gaianet/log/start-llamaedge.log 
[2024-07-24 19:28:49.693] [wasi_logging_stdout] [info] server_config: rag_api_server in src/main.rs:132: server_version: 0.7.4
[2024-07-24 19:28:49.693] [wasi_logging_stdout] [info] server_config: rag_api_server in src/main.rs:140: model_name: Phi-3-mini-4k-instruct,Nomic-embed-text-v1.5
[2024-07-24 19:28:49.693] [wasi_logging_stdout] [info] server_config: rag_api_server in src/main.rs:148: model_alias: default,embedding
[2024-07-24 19:28:49.694] [wasi_logging_stdout] [info] server_config: rag_api_server in src/main.rs:162: ctx_size: 4096,512
[2024-07-24 19:28:49.694] [wasi_logging_stdout] [info] server_config: rag_api_server in src/main.rs:176: batch_size: 4096,512
[2024-07-24 19:28:49.694] [wasi_logging_stdout] [info] server_config: rag_api_server in src/main.rs:190: prompt_template: phi-3-chat,embedding
[2024-07-24 19:28:49.695] [wasi_logging_stdout] [info] server_config: rag_api_server in src/main.rs:199: rag_prompt: You are a tour guide in Paris, France. Use information in the following context to directly answer the question from a Paris visitor.\n----------------\n
[2024-07-24 19:28:49.696] [wasi_logging_stdout] [info] server_config: rag_api_server in src/main.rs:220: qdrant_url: http://127.0.0.1:6333
[2024-07-24 19:28:49.696] [wasi_logging_stdout] [info] server_config: rag_api_server in src/main.rs:223: qdrant_collection_name: default
[2024-07-24 19:28:49.696] [wasi_logging_stdout] [info] server_config: rag_api_server in src/main.rs:226: qdrant_limit: 1
[2024-07-24 19:28:49.696] [wasi_logging_stdout] [info] server_config: rag_api_server in src/main.rs:229: qdrant_score_threshold: 0.5
[2024-07-24 19:28:49.696] [wasi_logging_stdout] [info] server_config: rag_api_server in src/main.rs:240: chunk_capacity: 100
[2024-07-24 19:28:49.697] [wasi_logging_stdout] [info] server_config: rag_api_server in src/main.rs:243: rag_policy: system-message
[2024-07-24 19:28:49.697] [wasi_logging_stdout] [info] llama-core: llama_core in /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/llama-core-0.13.0/src/lib.rs:496: Initializing the core context for RAG scenarios
[2024-07-24 19:28:49.698] [info] [WASI-NN] GGML backend: LLAMA_COMMIT 751fcfc6
[2024-07-24 19:28:49.698] [info] [WASI-NN] GGML backend: LLAMA_BUILD_NUMBER 3445
juntao commented 1 month ago

Thank you. Support for older CPUs is more complicated than we thought. We need this PR to be merged on WasmEdge to try again. Thanks.

https://github.com/WasmEdge/WasmEdge/pull/3597

OnlyDeniko commented 1 month ago

Thank you. Support for older CPUs is more complicated than we thought. We need this PR to be merged on WasmEdge to try again. Thanks.

WasmEdge/WasmEdge#3597

@juntao Hello there. PR has been merged. what are the startup instructions for the old CPUs now?