search

kth8 / bitnet

Run BitNet LLM in a container
Apache License 2.0
9 stars 3 forks source link

GPU issue ? #3

Closed alexmodesto73 closed 2 weeks ago

alexmodesto73 commented 2 weeks ago

cannot run :

Digest: sha256:de199a88ac62acf04daa7940e36da495716ff6224e2d6ff588598745417ae05a Status: Downloaded newer image for ghcr.io/kth8/bitnet:latest warning: not compiled with GPU offload support, --gpu-layers option will be ignored warning: see main README.md for information on enabling GPU BLAS support build: 0 (unknown) with Alpine clang version 17.0.6 for x86_64-alpine-linux-musl main: llama backend init Error occurred while running command: Command '['build/bin/llama-cli', '-m', 'Llama3-8B-1.58-100B-tokens-TQ2_0.gguf', '-n', '128', '-t', '2', '-p', 'The meaning to life and the universe is', '-ngl', '0', '-c', '2048', '--temp', '0.8', '-b', '1']' died with <Signals.SIGILL: 4>.

kth8 commented 2 weeks ago

@alexmodesto73 This requires a CPU with AVX2 support from Intel Haswell/AMD Excavator or later generations.

alexmodesto73 commented 2 weeks ago

ok :+1: