tluyben
commented
10 months ago No you don't need a GPU to run; it's made for CPU. I have seen this issue before on and it seems to have to do something with permissions. Can you see if the file is actually there or not, because for other people who reported it seems the file is there but from inside docker it's not accessible.
I get this error on running the project on a NixOS machine with no GPU, does this need an NVIDIA gpu to run?
1655000K .......... .......... .......... .......... .......... 39% 21.8M 1m48s 1655050K .......... .......... .......... .......... .......... 39% 24.3M 1m48s 1655100K .......... .......... .......... .......... .......... 39% 24.6M 1m48s 1655150K .......... .......... .......... .......... .......... 39% 24.3M 1m48s 1655200K .......... .......... .......... .......... .......... 39% 24.2M 1m48s 1655250K .......... .......... .......... .......... .......... 39% 24.6M 1m48s 1655300K .......... .......... .......... .......... .......... 39% 24.1M 1m48s 1655350K .......... .......... .......... .......... .......... 39% 18.0M 1m48s 1655400K .......... .......... .......... .......... .......... 39% 24.3M 1m48s 1655450K .......... .......... .......... .......... .......... 39% 24.6M 1m48s 1655500K .......... .......... .......... .......... .......... 39% 22.5M 1m48s 1655550K .......... .......... .......... .......... .......... 39% 20.2M 1m48s 1655600K .......warning: couldn't find nvcc (nvidia c compiler) try setting $CUDA_PATH if it's installed {"timestamp":1701335522,"level":"INFO","function":"main","line":2258,"message":"build info","build":1500,"commit":"a30b324"} error: failed to open models/7B/ggml-model-f16.gguf: No such file or directory {"timestamp":1701335522,"level":"INFO","function":"main","line":2261,"message":"system info","n_threads":4,"n_threads_batch":-1,"total_threads":8,"system_info":"AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | "} warning: couldn't find nvcc (nvidia c compiler) try setting $CUDA_PATH if it's installed {"timestamp":1701335572,"level":"INFO","function":"main","line":2258,"message":"build info","build":1500,"commit":"a30b324"} {"timestamp":1701335572,"level":"INFO","function":"main","line":2261,"message":"system info","n_threads":4,"n_threads_batch":-1,"total_threads":8,"system_info":"AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | "} error: failed to open models/7B/ggml-model-f16.gguf: No such file or directory