search

microsoft / BitNet

Official inference framework for 1-bit LLMs
MIT License
11.21k stars 760 forks source link

e2e_benchmark.py uses incorrect build path? #97

Closed grctest closed 2 weeks ago

grctest commented 2 weeks ago

I tried using the readme instructions for running the benchmark in Windows 11, however I got the following warning:

python .\utils\e2e_benchmark.py -m /models/Llama3-8B-1.58-100B-tokens/ggml-model-i2_s.gguf -n 200 -p 256 -t 4
ERROR:root:Benchmark binary not found, please build first.

The file does exist at BitNet\build\bin\Release\llama-bench.exe, however the e2e_benchmark.py file seems to reference an incorrect path:

bench_path = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "build/bin/llama-bench")

The run_inference.py file works as expected, and uses the following file path:

mainPath = path.join(buildDir, 'bin', 'Release', 'llama-cli.exe');

potassiummmm commented 2 weeks ago

Thanks for reporting! Fixed in https://github.com/microsoft/BitNet/commit/80b94aecb2079c88bad6b3164f7c13e58124a52a.