ggerganov / llama.cpp

LLM inference in C/C++

MIT License

66.02k stars 9.48k forks source link

Illegal instruction on Android (Honor Magic 5) #3622

Closed theoctopusride closed 11 months ago

theoctopusride commented 12 months ago

Prerequisites

Please answer the following questions for yourself before submitting an issue.

[x] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
[x] I carefully followed the README.md.
[x] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
[x] I reviewed the Discussions, and have a new bug or useful enhancement to share.

Current Behavior

$ git clone --depth 1 https://github.com/ggerganov/llama.cpp
$ cd llama.cpp
$ mkdir -p build
$ rm -rf build/*
$ cd build
$ cmake .. -DLLAMA_SANITIZE_ADDRESS=ON && cmake --build . --config Debug

-- CMAKE_SYSTEM_PROCESSOR: aarch64
-- ARM detected
-- Configuring done (0.3s)
-- Generating done (0.1s)
-- Build files have been written to: /data/data/com.termux/files/home/llama.cpp/build
[  1%] Built target BUILD_INFO
[  2%] Building C object CMakeFiles/ggml.dir/ggml.c.o
/data/data/com.termux/files/home/llama.cpp/ggml.c:2432:5: warning: implicit conversion increases floating-point precision: 'float32_t' (aka 'float') to 'ggml_float' (aka 'double') [-Wdouble-promotion]
 2432 |     GGML_F16_VEC_REDUCE(sumf, sum);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/data/data/com.termux/files/home/llama.cpp/ggml.c:1959:41: note: expanded from macro 'GGML_F16_VEC_REDUCE'
 1959 |     #define GGML_F16_VEC_REDUCE         GGML_F32Cx4_REDUCE
      |                                         ^
/data/data/com.termux/files/home/llama.cpp/ggml.c:1949:38: note: expanded from macro 'GGML_F32Cx4_REDUCE'
 1949 |     #define GGML_F32Cx4_REDUCE       GGML_F32x4_REDUCE
      |                                      ^
/data/data/com.termux/files/home/llama.cpp/ggml.c:1879:11: note: expanded from macro 'GGML_F32x4_REDUCE'
 1879 |     res = GGML_F32x4_REDUCE_ONE(x[0]);         \
      |         ~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~
/data/data/com.termux/files/home/llama.cpp/ggml.c:1864:34: note: expanded from macro 'GGML_F32x4_REDUCE_ONE'
 1864 | #define GGML_F32x4_REDUCE_ONE(x) vaddvq_f32(x)
      |                                  ^~~~~~~~~~~~~
/data/data/com.termux/files/home/llama.cpp/ggml.c:3692:9: warning: implicit conversion increases floating-point precision: 'float32_t' (aka 'float') to 'ggml_float' (aka 'double') [-Wdouble-promotion]
 3692 |         GGML_F16_VEC_REDUCE(sumf[k], sum[k]);
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/data/data/com.termux/files/home/llama.cpp/ggml.c:1959:41: note: expanded from macro 'GGML_F16_VEC_REDUCE'
 1959 |     #define GGML_F16_VEC_REDUCE         GGML_F32Cx4_REDUCE
      |                                         ^
/data/data/com.termux/files/home/llama.cpp/ggml.c:1949:38: note: expanded from macro 'GGML_F32Cx4_REDUCE'
 1949 |     #define GGML_F32Cx4_REDUCE       GGML_F32x4_REDUCE
      |                                      ^
/data/data/com.termux/files/home/llama.cpp/ggml.c:1879:11: note: expanded from macro 'GGML_F32x4_REDUCE'
 1879 |     res = GGML_F32x4_REDUCE_ONE(x[0]);         \
      |         ~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~
/data/data/com.termux/files/home/llama.cpp/ggml.c:1864:34: note: expanded from macro 'GGML_F32x4_REDUCE_ONE'
 1864 | #define GGML_F32x4_REDUCE_ONE(x) vaddvq_f32(x)
      |                                  ^~~~~~~~~~~~~
2 warnings generated.
[  3%] Building C object CMakeFiles/ggml.dir/ggml-alloc.c.o
[  4%] Building C object CMakeFiles/ggml.dir/ggml-backend.c.o
[  5%] Building C object CMakeFiles/ggml.dir/k_quants.c.o
[  5%] Built target ggml
[  6%] Linking C static library libggml_static.a
[  6%] Built target ggml_static
[  7%] Building CXX object CMakeFiles/llama.dir/llama.cpp.o
[  8%] Linking CXX static library libllama.a
[  8%] Built target llama
[ 10%] Building CXX object common/CMakeFiles/common.dir/common.cpp.o
[ 11%] Building CXX object common/CMakeFiles/common.dir/sampling.cpp.o
[ 12%] Building CXX object common/CMakeFiles/common.dir/console.cpp.o
[ 13%] Building CXX object common/CMakeFiles/common.dir/grammar-parser.cpp.o
[ 14%] Building CXX object common/CMakeFiles/common.dir/train.cpp.o
[ 14%] Built target common
[ 15%] Building CXX object tests/CMakeFiles/test-quantize-fns.dir/test-quantize-fns.cpp.o
[ 16%] Linking CXX executable ../bin/test-quantize-fns
[ 16%] Built target test-quantize-fns
[ 17%] Building CXX object tests/CMakeFiles/test-quantize-perf.dir/test-quantize-perf.cpp.o
[ 19%] Linking CXX executable ../bin/test-quantize-perf
[ 19%] Built target test-quantize-perf
[ 20%] Building CXX object tests/CMakeFiles/test-sampling.dir/test-sampling.cpp.o
[ 21%] Linking CXX executable ../bin/test-sampling
[ 21%] Built target test-sampling
[ 22%] Building CXX object tests/CMakeFiles/test-tokenizer-0-llama.dir/test-tokenizer-0-llama.cpp.o
[ 23%] Linking CXX executable ../bin/test-tokenizer-0-llama
[ 23%] Built target test-tokenizer-0-llama
[ 24%] Building CXX object tests/CMakeFiles/test-tokenizer-0-falcon.dir/test-tokenizer-0-falcon.cpp.o
[ 25%] Linking CXX executable ../bin/test-tokenizer-0-falcon
[ 25%] Built target test-tokenizer-0-falcon
[ 26%] Building CXX object tests/CMakeFiles/test-tokenizer-1-llama.dir/test-tokenizer-1-llama.cpp.o
[ 28%] Linking CXX executable ../bin/test-tokenizer-1-llama
[ 28%] Built target test-tokenizer-1-llama
[ 29%] Building CXX object tests/CMakeFiles/test-tokenizer-1-bpe.dir/test-tokenizer-1-bpe.cpp.o
[ 30%] Linking CXX executable ../bin/test-tokenizer-1-bpe
[ 30%] Built target test-tokenizer-1-bpe
[ 31%] Building CXX object tests/CMakeFiles/test-grammar-parser.dir/test-grammar-parser.cpp.o
[ 32%] Linking CXX executable ../bin/test-grammar-parser
[ 32%] Built target test-grammar-parser
[ 33%] Building CXX object tests/CMakeFiles/test-llama-grammar.dir/test-llama-grammar.cpp.o
[ 34%] Linking CXX executable ../bin/test-llama-grammar
[ 34%] Built target test-llama-grammar
[ 35%] Building CXX object tests/CMakeFiles/test-grad0.dir/test-grad0.cpp.o
[ 37%] Linking CXX executable ../bin/test-grad0
[ 37%] Built target test-grad0
[ 38%] Building CXX object tests/CMakeFiles/test-rope.dir/test-rope.cpp.o
[ 39%] Linking CXX executable ../bin/test-rope
[ 39%] Built target test-rope
[ 40%] Building C object tests/CMakeFiles/test-c.dir/test-c.c.o
[ 41%] Linking CXX executable ../bin/test-c
[ 41%] Built target test-c
[ 42%] Building CXX object examples/main/CMakeFiles/main.dir/main.cpp.o
[ 43%] Linking CXX executable ../../bin/main
[ 43%] Built target main
[ 44%] Building CXX object examples/quantize/CMakeFiles/quantize.dir/quantize.cpp.o
[ 46%] Linking CXX executable ../../bin/quantize
[ 46%] Built target quantize
[ 47%] Building CXX object examples/quantize-stats/CMakeFiles/quantize-stats.dir/quantize-stats.cpp.o
[ 48%] Linking CXX executable ../../bin/quantize-stats
[ 48%] Built target quantize-stats
[ 49%] Building CXX object examples/perplexity/CMakeFiles/perplexity.dir/perplexity.cpp.o
[ 50%] Linking CXX executable ../../bin/perplexity
[ 50%] Built target perplexity
[ 51%] Building CXX object examples/embedding/CMakeFiles/embedding.dir/embedding.cpp.o
[ 52%] Linking CXX executable ../../bin/embedding
[ 52%] Built target embedding
[ 53%] Building CXX object examples/save-load-state/CMakeFiles/save-load-state.dir/save-load-state.cpp.o
[ 55%] Linking CXX executable ../../bin/save-load-state
[ 55%] Built target save-load-state
[ 56%] Building CXX object examples/benchmark/CMakeFiles/benchmark.dir/benchmark-matmult.cpp.o
[ 57%] Linking CXX executable ../../bin/benchmark
[ 57%] Built target benchmark
[ 58%] Building CXX object examples/baby-llama/CMakeFiles/baby-llama.dir/baby-llama.cpp.o
[ 59%] Linking CXX executable ../../bin/baby-llama
[ 59%] Built target baby-llama
[ 60%] Building CXX object examples/train-text-from-scratch/CMakeFiles/train-text-from-scratch.dir/train-text-from-scratch.cpp.o
[ 61%] Linking CXX executable ../../bin/train-text-from-scratch
[ 61%] Built target train-text-from-scratch
[ 62%] Building CXX object examples/finetune/CMakeFiles/finetune.dir/finetune.cpp.o
[ 64%] Linking CXX executable ../../bin/finetune
[ 64%] Built target finetune
[ 65%] Building CXX object examples/convert-llama2c-to-ggml/CMakeFiles/convert-llama2c-to-ggml.dir/convert-llama2c-to-ggml.cpp.o
[ 66%] Linking CXX executable ../../bin/convert-llama2c-to-ggml
[ 66%] Built target convert-llama2c-to-ggml
[ 67%] Building CXX object examples/simple/CMakeFiles/simple.dir/simple.cpp.o
[ 68%] Linking CXX executable ../../bin/simple
[ 68%] Built target simple
[ 69%] Building CXX object examples/batched/CMakeFiles/batched.dir/batched.cpp.o
[ 70%] Linking CXX executable ../../bin/batched
[ 70%] Built target batched
[ 71%] Building CXX object examples/batched-bench/CMakeFiles/batched-bench.dir/batched-bench.cpp.o
[ 73%] Linking CXX executable ../../bin/batched-bench
[ 73%] Built target batched-bench
[ 74%] Building CXX object examples/speculative/CMakeFiles/speculative.dir/speculative.cpp.o
[ 75%] Linking CXX executable ../../bin/speculative
[ 75%] Built target speculative
[ 76%] Building CXX object examples/parallel/CMakeFiles/parallel.dir/parallel.cpp.o
[ 77%] Linking CXX executable ../../bin/parallel
[ 77%] Built target parallel
[ 78%] Building CXX object examples/embd-input/CMakeFiles/embdinput.dir/embd-input-lib.cpp.o
[ 79%] Linking CXX static library libembdinput.a
[ 79%] Built target embdinput
[ 80%] Building CXX object examples/embd-input/CMakeFiles/embd-input-test.dir/embd-input-test.cpp.o
[ 82%] Linking CXX executable ../../bin/embd-input-test
[ 82%] Built target embd-input-test
[ 83%] Building CXX object examples/llava/CMakeFiles/clip.dir/clip.cpp.o
[ 84%] Linking CXX static library libclip.a
[ 84%] Built target clip
[ 85%] Building CXX object examples/llava/CMakeFiles/llava.dir/llava.cpp.o
[ 86%] Linking CXX executable ../../bin/llava
[ 86%] Built target llava
[ 87%] Building CXX object examples/llama-bench/CMakeFiles/llama-bench.dir/llama-bench.cpp.o
[ 88%] Linking CXX executable ../../bin/llama-bench
[ 88%] Built target llama-bench
[ 89%] Building CXX object examples/beam-search/CMakeFiles/beam-search.dir/beam-search.cpp.o
[ 91%] Linking CXX executable ../../bin/beam-search
[ 91%] Built target beam-search
[ 92%] Building CXX object examples/server/CMakeFiles/server.dir/server.cpp.o
[ 93%] Linking CXX executable ../../bin/server
[ 93%] Built target server
[ 94%] Building CXX object examples/export-lora/CMakeFiles/export-lora.dir/export-lora.cpp.o
[ 95%] Linking CXX executable ../../bin/export-lora
[ 95%] Built target export-lora
[ 96%] Building CXX object pocs/vdot/CMakeFiles/vdot.dir/vdot.cpp.o
[ 97%] Linking CXX executable ../../bin/vdot
[ 97%] Built target vdot
[ 98%] Building CXX object pocs/vdot/CMakeFiles/q8dot.dir/q8dot.cpp.o
[100%] Linking CXX executable ../../bin/q8dot
[100%] Built target q8dot

$ ./build/bin/main -m /sdcard/Download/llama-2-7b-chat.Q3_K_S.gguf -color -c 2048 --keep 1 -t 3 -b 10 -i -ins

Log start  
main: build = 1 (2a4bcba)  
main: built with clang version 17.0.2 for aarch64-unknown-linux-android24
main: seed  = 1697243692
llama_model_loader: loaded meta data with 19 key-value pairs and 291 tensors from /sdcard/Download/llama-2-7b-chat.Q3_K_S.gguf (version GGUF V2 (latest))
llama_model_loader: - tensor    0:                token_embd.weight q3_K     [  4096, 32000,     1,     1 ]
llama_model_loader: - tensor    1:           blk.0.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor    2:            blk.0.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor    3:            blk.0.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor    4:              blk.0.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor    5:            blk.0.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor    6:              blk.0.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor    7:         blk.0.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor    8:              blk.0.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor    9:              blk.0.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   10:           blk.1.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   11:            blk.1.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor   12:            blk.1.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   13:              blk.1.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   14:            blk.1.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   15:              blk.1.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   16:         blk.1.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   17:              blk.1.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   18:              blk.1.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   19:          blk.10.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   20:           blk.10.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor   21:           blk.10.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   22:             blk.10.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   23:           blk.10.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   24:             blk.10.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   25:        blk.10.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   26:             blk.10.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   27:             blk.10.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   28:          blk.11.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   29:           blk.11.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor   30:           blk.11.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   31:             blk.11.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   32:           blk.11.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   33:             blk.11.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   34:        blk.11.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   35:             blk.11.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   36:             blk.11.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   37:          blk.12.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   38:           blk.12.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor   39:           blk.12.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   40:             blk.12.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   41:           blk.12.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   42:             blk.12.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   43:        blk.12.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   44:             blk.12.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   45:             blk.12.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   46:          blk.13.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   47:           blk.13.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor   48:           blk.13.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   49:             blk.13.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   50:           blk.13.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   51:             blk.13.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   52:        blk.13.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   53:             blk.13.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   54:             blk.13.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   55:          blk.14.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   56:           blk.14.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor   57:           blk.14.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   58:             blk.14.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   59:           blk.14.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   60:             blk.14.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   61:        blk.14.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   62:             blk.14.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   63:             blk.14.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   64:          blk.15.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   65:           blk.15.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor   66:           blk.15.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   67:             blk.15.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   68:           blk.15.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   69:             blk.15.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   70:        blk.15.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   71:             blk.15.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   72:             blk.15.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   73:          blk.16.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   74:           blk.16.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor   75:           blk.16.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   76:             blk.16.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   77:           blk.16.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   78:             blk.16.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   79:        blk.16.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   80:             blk.16.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   81:             blk.16.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   82:          blk.17.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   83:           blk.17.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor   84:           blk.17.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   85:             blk.17.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   86:           blk.17.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   87:             blk.17.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   88:        blk.17.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   89:             blk.17.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   90:             blk.17.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   91:          blk.18.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   92:           blk.18.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor   93:           blk.18.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   94:             blk.18.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor   95:           blk.18.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor   96:             blk.18.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   97:        blk.18.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   98:             blk.18.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor   99:             blk.18.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  100:          blk.19.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  101:           blk.19.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  102:           blk.19.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  103:             blk.19.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  104:           blk.19.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  105:             blk.19.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  106:        blk.19.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  107:             blk.19.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  108:             blk.19.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  109:           blk.2.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  110:            blk.2.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  111:            blk.2.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  112:              blk.2.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  113:            blk.2.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  114:              blk.2.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  115:         blk.2.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  116:              blk.2.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  117:              blk.2.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  118:          blk.20.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  119:           blk.20.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  120:           blk.20.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  121:             blk.20.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  122:           blk.20.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  123:             blk.20.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  124:        blk.20.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  125:             blk.20.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  126:             blk.20.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  127:          blk.21.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  128:           blk.21.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  129:           blk.21.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  130:             blk.21.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  131:           blk.21.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  132:             blk.21.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  133:        blk.21.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  134:             blk.21.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  135:             blk.21.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  136:          blk.22.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  137:           blk.22.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  138:           blk.22.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  139:             blk.22.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  140:           blk.22.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  141:             blk.22.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  142:        blk.22.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  143:             blk.22.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  144:             blk.22.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  145:          blk.23.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  146:           blk.23.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  147:           blk.23.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  148:             blk.23.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  149:           blk.23.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  150:             blk.23.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  151:        blk.23.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  152:             blk.23.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  153:             blk.23.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  154:           blk.3.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  155:            blk.3.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  156:            blk.3.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  157:              blk.3.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  158:            blk.3.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  159:              blk.3.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  160:         blk.3.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  161:              blk.3.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  162:              blk.3.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  163:           blk.4.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  164:            blk.4.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  165:            blk.4.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  166:              blk.4.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  167:            blk.4.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  168:              blk.4.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  169:         blk.4.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  170:              blk.4.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  171:              blk.4.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  172:           blk.5.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  173:            blk.5.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  174:            blk.5.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  175:              blk.5.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  176:            blk.5.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  177:              blk.5.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  178:         blk.5.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  179:              blk.5.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  180:              blk.5.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  181:           blk.6.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  182:            blk.6.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  183:            blk.6.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  184:              blk.6.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  185:            blk.6.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  186:              blk.6.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  187:         blk.6.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  188:              blk.6.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  189:              blk.6.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  190:           blk.7.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  191:            blk.7.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  192:            blk.7.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  193:              blk.7.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  194:            blk.7.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  195:              blk.7.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  196:         blk.7.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  197:              blk.7.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  198:              blk.7.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  199:           blk.8.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  200:            blk.8.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  201:            blk.8.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  202:              blk.8.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  203:            blk.8.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  204:              blk.8.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  205:         blk.8.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  206:              blk.8.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  207:              blk.8.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  208:           blk.9.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  209:            blk.9.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  210:            blk.9.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  211:              blk.9.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  212:            blk.9.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  213:              blk.9.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  214:         blk.9.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  215:              blk.9.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  216:              blk.9.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  217:                    output.weight q6_K     [  4096, 32000,     1,     1 ]
llama_model_loader: - tensor  218:          blk.24.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  219:           blk.24.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  220:           blk.24.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  221:             blk.24.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  222:           blk.24.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  223:             blk.24.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  224:        blk.24.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  225:             blk.24.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  226:             blk.24.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  227:          blk.25.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  228:           blk.25.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  229:           blk.25.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  230:             blk.25.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  231:           blk.25.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  232:             blk.25.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  233:        blk.25.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  234:             blk.25.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  235:             blk.25.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  236:          blk.26.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  237:           blk.26.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  238:           blk.26.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  239:             blk.26.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  240:           blk.26.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  241:             blk.26.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  242:        blk.26.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  243:             blk.26.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  244:             blk.26.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  245:          blk.27.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  246:           blk.27.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  247:           blk.27.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  248:             blk.27.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  249:           blk.27.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  250:             blk.27.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  251:        blk.27.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  252:             blk.27.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  253:             blk.27.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  254:          blk.28.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  255:           blk.28.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  256:           blk.28.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  257:             blk.28.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  258:           blk.28.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  259:             blk.28.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  260:        blk.28.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  261:             blk.28.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  262:             blk.28.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  263:          blk.29.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  264:           blk.29.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  265:           blk.29.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  266:             blk.29.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  267:           blk.29.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  268:             blk.29.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  269:        blk.29.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  270:             blk.29.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  271:             blk.29.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  272:          blk.30.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  273:           blk.30.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  274:           blk.30.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  275:             blk.30.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  276:           blk.30.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  277:             blk.30.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  278:        blk.30.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  279:             blk.30.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  280:             blk.30.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  281:          blk.31.attn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  282:           blk.31.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]
llama_model_loader: - tensor  283:           blk.31.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  284:             blk.31.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]
llama_model_loader: - tensor  285:           blk.31.ffn_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - tensor  286:             blk.31.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  287:        blk.31.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  288:             blk.31.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  289:             blk.31.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]
llama_model_loader: - tensor  290:               output_norm.weight f32      [  4096,     1,     1,     1 ]
llama_model_loader: - kv   0:                       general.architecture str     
llama_model_loader: - kv   1:                               general.name str     
llama_model_loader: - kv   2:                       llama.context_length u32     
llama_model_loader: - kv   3:                     llama.embedding_length u32     
llama_model_loader: - kv   4:                          llama.block_count u32     
llama_model_loader: - kv   5:                  llama.feed_forward_length u32     
llama_model_loader: - kv   6:                 llama.rope.dimension_count u32     
llama_model_loader: - kv   7:                 llama.attention.head_count u32     
llama_model_loader: - kv   8:              llama.attention.head_count_kv u32     
llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32     
llama_model_loader: - kv  10:                          general.file_type u32     
llama_model_loader: - kv  11:                       tokenizer.ggml.model str     
llama_model_loader: - kv  12:                      tokenizer.ggml.tokens arr     
llama_model_loader: - kv  13:                      tokenizer.ggml.scores arr     
llama_model_loader: - kv  14:                  tokenizer.ggml.token_type arr     
llama_model_loader: - kv  15:                tokenizer.ggml.bos_token_id u32     
llama_model_loader: - kv  16:                tokenizer.ggml.eos_token_id u32     
llama_model_loader: - kv  17:            tokenizer.ggml.unknown_token_id u32     
llama_model_loader: - kv  18:               general.quantization_version u32     
llama_model_loader: - type  f32:   65 tensors
llama_model_loader: - type q3_K:  225 tensors
llama_model_loader: - type q6_K:    1 tensors
llm_load_print_meta: format           = GGUF V2 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = SPM
llm_load_print_meta: n_vocab          = 32000
llm_load_print_meta: n_merges         = 0
llm_load_print_meta: n_ctx_train      = 4096
llm_load_print_meta: n_embd           = 4096
llm_load_print_meta: n_head           = 32
llm_load_print_meta: n_head_kv        = 32
llm_load_print_meta: n_layer          = 32
llm_load_print_meta: n_rot            = 128
llm_load_print_meta: n_gqa            = 1
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-06
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: n_ff             = 11008
llm_load_print_meta: freq_base_train  = 10000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: model type       = 7B
llm_load_print_meta: model ftype      = mostly Q3_K - Small
llm_load_print_meta: model params     = 6.74 B
llm_load_print_meta: model size       = 2.75 GiB (3.50 BPW) 
llm_load_print_meta: general.name   = LLaMA v2
llm_load_print_meta: BOS token = 1 '<s>'
llm_load_print_meta: EOS token = 2 '</s>'
llm_load_print_meta: UNK token = 0 '<unk>'
llm_load_print_meta: LF token  = 13 '<0x0A>'
llm_load_tensors: ggml ctx size =    0.10 MB
llm_load_tensors: mem required  = 2811.11 MB
.................................................................................................
llama_new_context_with_model: n_ctx      = 2048
llama_new_context_with_model: freq_base  = 10000.0
llama_new_context_with_model: freq_scale = 1
llama_new_context_with_model: kv self size  = 1024.00 MB
llama_new_context_with_model: compute buffer total size = 9.17 MB
Illegal instruction

Environment and Context

Please provide detailed information about your computer setup. This is important in case the issue is not reproducible except for under certain specific conditions.

Physical (or virtual) hardware you are using, e.g. for Linux:

$ lscpu

Architecture:                    aarch64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
CPU(s):                          8
On-line CPU(s) list:             0-7
Vendor ID:                       ARM
Model name:                      Cortex-A510
Model:                           1
Thread(s) per core:              1
Core(s) per socket:              3
Socket(s):                       1
Stepping:                        r1p1
Frequency boost:                 enabled
CPU(s) scaling MHz:              61%
CPU max MHz:                     2016.0000
CPU min MHz:                     307.2000
BogoMIPS:                        38.40
Flags:                           fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 asimdfhm dit uscat ilrcpc flagm ssbs sb paca pacg dcpodp flagm2 frint i8mm bf16 bti
Model name:                      Cortex-A715
Model:                           0
Thread(s) per core:              1
Core(s) per socket:              2
Socket(s):                       1
Stepping:                        r1p0
CPU(s) scaling MHz:              59%
CPU max MHz:                     2803.2000
CPU min MHz:                     499.2000
BogoMIPS:                        38.40
Flags:                           fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 asimdfhm dit uscat ilrcpc flagm ssbs sb paca pacg dcpodp flagm2 frint i8mm bf16 bti
Model name:                      Cortex-A710
Model:                           0
Thread(s) per core:              1
Core(s) per socket:              2
Socket(s):                       1
Stepping:                        r2p0
CPU(s) scaling MHz:              59%
CPU max MHz:                     2803.2000
CPU min MHz:                     499.2000
BogoMIPS:                        38.40
Flags:                           fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 asimdfhm dit uscat ilrcpc flagm ssbs sb paca pacg dcpodp flagm2 frint i8mm bf16 bti
Model name:                      -
Model:                           0
Thread(s) per core:              1
Core(s) per socket:              1
Socket(s):                       1
Stepping:                        0x1
CPU(s) scaling MHz:              27%
CPU max MHz:                     3187.2000
CPU min MHz:                     595.2000
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:        Mitigation; __user pointer sanitization
Vulnerability Spectre v2:        Vulnerable: Unprivileged eBPF enabled
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected

Operating System, e.g. for Linux:

$ uname -a

Linux localhost 5.15.41-qki-consolidate-android13-8-g8d73f5ad0193 #1 SMP PREEMPT Tue Dec 27 04:30:59 UTC 2022 aarch64 Android

$ python3 --version
Python 3.11.6
$ make --version
GNU Make 4.4.1
$ cmake --version
cmake version 3.27.7
$ g++ --version
clang version 17.0.2
Target: aarch64-unknown-linux-android24
Thread model: posix
InstalledDir: /data/data/com.termux/files/usr/bin

Failure Information (for bugs)

see above

Steps to Reproduce

see above

Failure Logs

see above

ggerganov commented 12 months ago

Can you get a stack trace when it crashes?

staviq commented 12 months ago

Can you get a stack trace when it crashes?

@theoctopusride Run it with gdb --args ./build/bin/main -m /sdcard/Download/llama-2-7b-chat.Q3_K_S.gguf -color -c 2048 --keep 1 -t 3 -b 10 -i -ins

When gdb starts type r (enter) to start

When it crashes, type bt (enter) and post the output.

ElvisClaros commented 12 months ago

Hi, I have a similar problem.

➜  llama.cpp git:(master) ✗ make LLAMA_CLBLAST=1 LLAMA_DEBUG=1 -j8
I llama.cpp build info:
I UNAME_S:   Linux
I UNAME_P:   unknown
I UNAME_M:   aarch64
I CFLAGS:    -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c11   -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -pthread -mcpu=native
I CXXFLAGS:  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi
I NVCCFLAGS:  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native     -Wno-pedantic -Xcompiler "-Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi "
I LDFLAGS:   -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
I CC:        clang version 17.0.2
I CXX:       clang version 17.0.2

cc  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c11   -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -pthread -mcpu=native     -c ggml.c -o ggml.o
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  -c llama.cpp -o llama.o
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  -c common/console.cpp -o console.o
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  -c common/grammar-parser.cpp -o grammar-parser.o
cc -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c11   -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -pthread -mcpu=native   -c k_quants.c -o k_quants.o
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  -c ggml-opencl.cpp -o ggml-opencl.o
cc  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c11   -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -pthread -mcpu=native     -c ggml-alloc.c -o ggml-alloc.o
cc  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c11   -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -pthread -mcpu=native     -c ggml-backend.c -o ggml-backend.o
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  -c common/train.cpp -o train.o
cc -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c11   -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -pthread -mcpu=native   -c tests/test-c.c -o tests/test-c.o
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  -c common/common.cpp -o common.o
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  -c common/sampling.cpp -o sampling.o
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  pocs/vdot/vdot.cpp ggml.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o vdot -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  pocs/vdot/q8dot.cpp ggml.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o q8dot -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/benchmark/benchmark-matmult.cpp ggml.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o benchmark-matmult -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/main/main.cpp ggml.o llama.o common.o sampling.o console.o grammar-parser.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o main -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/quantize/quantize.cpp ggml.o llama.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o quantize -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/quantize-stats/quantize-stats.cpp ggml.o llama.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o quantize-stats -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/perplexity/perplexity.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o perplexity -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/embedding/embedding.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o embedding -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/train-text-from-scratch/train-text-from-scratch.cpp ggml.o llama.o common.o sampling.o train.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o train-text-from-scratch -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp ggml.o llama.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o convert-llama2c-to-ggml -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/simple/simple.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o simple -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/batched/batched.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o batched -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/batched-bench/batched-bench.cpp ggml.o llama.o common.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o batched-bench -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/save-load-state/save-load-state.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o save-load-state -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  -Iexamples/server examples/server/server.cpp ggml.o llama.o common.o sampling.o grammar-parser.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o server -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ --shared -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/embd-input/embd-input-lib.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o libembdinput.so -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/gguf/gguf.cpp ggml.o llama.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o gguf -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL

====  Run ./main -h for help.  ====

aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/llama-bench/llama-bench.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o llama-bench -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/llava/llava.cpp examples/llava/clip.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o llava -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL  -Wno-cast-qual
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/baby-llama/baby-llama.cpp ggml.o llama.o common.o sampling.o train.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o baby-llama -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/beam-search/beam-search.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o beam-search -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/speculative/speculative.cpp ggml.o llama.o common.o sampling.o grammar-parser.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o speculative -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/infill/infill.cpp ggml.o llama.o common.o sampling.o console.o grammar-parser.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o infill -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/parallel/parallel.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o parallel -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/finetune/finetune.cpp ggml.o llama.o common.o sampling.o train.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o finetune -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/export-lora/export-lora.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o export-lora -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread -mcpu=native   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/embd-input/embd-input-test.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o embd-input-test -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL  -L. -lembdinput

➜  llama.cpp git:(master) ✗ lscpu                 (10-14 11:09)
Architecture:           aarch64
  CPU op-mode(s):       32-bit, 64-bit
  Byte Order:           Little Endian
CPU(s):                 8
  On-line CPU(s) list:  0-7
Vendor ID:              Qualcomm
  Model name:           Kryo-3XX-Silver
    Model:              12
    Thread(s) per core: 1
    Core(s) per socket: 6
    Socket(s):          1
    Stepping:           0x7
    CPU(s) scaling MHz: 78%
    CPU max MHz:        1708.8000
    CPU min MHz:        300.0000
    BogoMIPS:           38.00
    Flags:              fp asimd evtstrm aes pmull sha1 sha2 crc
                        32 atomics fphp asimdhp
  Model name:           Kryo-3XX-Gold
    Model:              13
    Thread(s) per core: 1
    Core(s) per socket: 2
    Socket(s):          1
    Stepping:           0x6
    CPU(s) scaling MHz: 70%
    CPU max MHz:        2208.0000
    CPU min MHz:        300.0000
    BogoMIPS:           38.00
    Flags:              fp asimd evtstrm aes pmull sha1 sha2 crc
                        32 atomics fphp asimdhp
Caches (sum of all):
  L1d:                  320 KiB (8 instances)
  L1i:                  320 KiB (8 instances)
  L2:                   1.3 MiB (8 instances)
  L3:                   1 MiB (1 instance)

➜  llama.cpp git:(master) ✗ uname -a              (10-14 11:09)
Linux localhost 4.9.227-perf+ #1 SMP PREEMPT Thu May 19 09:53:49 CDT 2022 aarch64 Android

➜  llama.cpp git:(master) ✗ python3 --version     (10-14 11:11)
Python 3.11.6

➜  llama.cpp git:(master) ✗ make --version        (10-14 11:12) GNU Make 4.4.1
Built for aarch64-unknown-linux-android
Copyright (C) 1988-2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

➜  llama.cpp git:(master) ✗ g++ --version         (10-14 11:12)
clang version 17.0.2
Target: aarch64-unknown-linux-android24
Thread model: posix
InstalledDir: /data/data/com.termux/files/usr/bin

➜  llama.cpp git:(master) ✗ gdb ./main            (10-14 11:14)
GNU gdb (GDB) 13.2
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-linux-android".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./main...
(gdb) r -m /data/data/com.termux/files/home/llama.cpp/models/tinyllama-1.1b-chat-v0.3.Q4_0.gguf -f ./prompts/tinyllama-chat.txt -r "<|im_start|>assistant" -r "<|im_start|>" -r "<|im_end|>" --in-suffix "<|im_end|>" --color -c 2048 -ins --temp 0.7 --repeat_penalty 1.1 -t 1 -n -1 -s -1
Starting program: /data/data/com.termux/files/home/llama.cpp/main -m /data/data/com.termux/files/home/llama.cpp/models/tinyllama-1.1b-chat-v0.3.Q4_0.gguf -f ./prompts/tinyllama-chat.txt -r "<|im_start|>assistant" -r "<|im_start|>" -r "<|im_end|>" --in-suffix "<|im_end|>" --color -c 2048 -ins --temp 0.7 --repeat_penalty 1.1 -t 1 -n -1 -s -1
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/data/data/com.termux/files/usr/lib/libthread_db.so".
Log start
main: build = 1381 (11dc109)
main: built with clang version 17.0.2 for aarch64-unknown-linux-android24
main: seed  = 1697292949
ggml_opencl: selecting platform: 'QUALCOMM Snapdragon(TM)'
ggml_opencl: selecting device: 'QUALCOMM Adreno(TM)'
ggml_opencl: device FP16 support: true
[New Thread 0x1ffc (LWP 8188)]
[New Thread 0x1ffd (LWP 8189)]
llama_model_loader: loaded meta data with 20 key-value pairs and 201 tensors from /data/data/com.termux/files/home/llama.cpp/models/tinyllama-1.1b-chat-v0.3.Q4_0.gguf (version GGUF V2 (latest))
llama_model_loader: - tensor    0:                token_embd.weight q4_0     [  2048, 32003,     1,     1 ]
llama_model_loader: - tensor    1:              blk.0.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor    2:              blk.0.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor    3:              blk.0.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor    4:         blk.0.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor    5:            blk.0.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor    6:              blk.0.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor    7:            blk.0.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor    8:           blk.0.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor    9:            blk.0.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   10:              blk.1.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   11:              blk.1.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   12:              blk.1.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   13:         blk.1.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   14:            blk.1.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   15:              blk.1.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   16:            blk.1.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   17:           blk.1.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   18:            blk.1.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   19:              blk.2.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   20:              blk.2.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   21:              blk.2.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   22:         blk.2.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   23:            blk.2.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   24:              blk.2.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   25:            blk.2.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   26:           blk.2.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   27:            blk.2.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   28:              blk.3.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   29:              blk.3.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   30:              blk.3.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   31:         blk.3.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   32:            blk.3.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   33:              blk.3.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   34:            blk.3.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   35:           blk.3.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   36:            blk.3.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   37:              blk.4.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   38:              blk.4.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   39:              blk.4.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   40:         blk.4.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   41:            blk.4.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   42:              blk.4.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   43:            blk.4.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   44:           blk.4.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   45:            blk.4.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   46:              blk.5.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   47:              blk.5.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   48:              blk.5.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   49:         blk.5.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   50:            blk.5.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   51:              blk.5.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   52:            blk.5.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   53:           blk.5.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   54:            blk.5.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   55:              blk.6.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   56:              blk.6.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   57:              blk.6.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   58:         blk.6.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   59:            blk.6.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   60:              blk.6.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   61:            blk.6.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   62:           blk.6.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   63:            blk.6.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   64:              blk.7.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   65:              blk.7.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   66:              blk.7.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   67:         blk.7.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   68:            blk.7.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   69:              blk.7.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   70:            blk.7.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   71:           blk.7.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   72:            blk.7.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   73:              blk.8.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   74:              blk.8.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   75:              blk.8.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   76:         blk.8.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   77:            blk.8.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   78:              blk.8.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   79:            blk.8.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   80:           blk.8.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   81:            blk.8.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   82:              blk.9.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   83:              blk.9.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   84:              blk.9.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   85:         blk.9.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   86:            blk.9.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   87:              blk.9.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   88:            blk.9.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   89:           blk.9.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   90:            blk.9.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   91:             blk.10.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   92:             blk.10.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   93:             blk.10.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   94:        blk.10.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   95:           blk.10.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   96:             blk.10.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   97:           blk.10.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   98:          blk.10.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   99:           blk.10.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  100:             blk.11.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  101:             blk.11.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  102:             blk.11.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  103:        blk.11.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  104:           blk.11.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  105:             blk.11.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  106:           blk.11.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  107:          blk.11.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  108:           blk.11.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  109:             blk.12.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  110:             blk.12.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  111:             blk.12.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  112:        blk.12.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  113:           blk.12.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  114:             blk.12.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  115:           blk.12.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  116:          blk.12.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  117:           blk.12.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  118:             blk.13.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  119:             blk.13.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  120:             blk.13.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  121:        blk.13.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  122:           blk.13.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  123:             blk.13.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  124:           blk.13.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  125:          blk.13.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  126:           blk.13.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  127:             blk.14.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  128:             blk.14.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  129:             blk.14.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  130:        blk.14.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  131:           blk.14.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  132:             blk.14.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  133:           blk.14.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  134:          blk.14.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  135:           blk.14.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  136:             blk.15.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  137:             blk.15.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  138:             blk.15.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  139:        blk.15.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  140:           blk.15.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  141:             blk.15.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  142:           blk.15.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  143:          blk.15.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  144:           blk.15.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  145:             blk.16.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  146:             blk.16.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  147:             blk.16.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  148:        blk.16.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  149:           blk.16.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  150:             blk.16.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  151:           blk.16.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  152:          blk.16.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  153:           blk.16.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  154:             blk.17.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  155:             blk.17.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  156:             blk.17.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  157:        blk.17.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  158:           blk.17.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  159:             blk.17.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  160:           blk.17.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  161:          blk.17.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  162:           blk.17.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  163:             blk.18.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  164:             blk.18.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  165:             blk.18.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  166:        blk.18.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  167:           blk.18.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  168:             blk.18.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  169:           blk.18.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  170:          blk.18.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  171:           blk.18.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  172:             blk.19.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  173:             blk.19.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  174:             blk.19.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  175:        blk.19.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  176:           blk.19.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  177:             blk.19.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  178:           blk.19.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  179:          blk.19.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  180:           blk.19.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  181:             blk.20.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  182:             blk.20.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  183:             blk.20.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  184:        blk.20.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  185:           blk.20.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  186:             blk.20.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  187:           blk.20.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  188:          blk.20.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  189:           blk.20.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  190:             blk.21.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  191:             blk.21.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  192:             blk.21.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  193:        blk.21.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  194:           blk.21.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  195:             blk.21.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  196:           blk.21.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  197:          blk.21.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  198:           blk.21.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  199:               output_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  200:                    output.weight q6_K     [  2048, 32003,     1,     1 ]
llama_model_loader: - kv   0:                       general.architecture str
llama_model_loader: - kv   1:                               general.name str
llama_model_loader: - kv   2:                       llama.context_length u32
llama_model_loader: - kv   3:                     llama.embedding_length u32
llama_model_loader: - kv   4:                          llama.block_count u32
llama_model_loader: - kv   5:                  llama.feed_forward_length u32
llama_model_loader: - kv   6:                 llama.rope.dimension_count u32
llama_model_loader: - kv   7:                 llama.attention.head_count u32
llama_model_loader: - kv   8:              llama.attention.head_count_kv u32
llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32
llama_model_loader: - kv  10:                       llama.rope.freq_base f32
llama_model_loader: - kv  11:                          general.file_type u32
llama_model_loader: - kv  12:                       tokenizer.ggml.model str
llama_model_loader: - kv  13:                      tokenizer.ggml.tokens arr
llama_model_loader: - kv  14:                      tokenizer.ggml.scores arr
llama_model_loader: - kv  15:                  tokenizer.ggml.token_type arr
llama_model_loader: - kv  16:                tokenizer.ggml.bos_token_id u32
llama_model_loader: - kv  17:                tokenizer.ggml.eos_token_id u32
llama_model_loader: - kv  18:            tokenizer.ggml.unknown_token_id u32
llama_model_loader: - kv  19:               general.quantization_version u32
llama_model_loader: - type  f32:   45 tensors
llama_model_loader: - type q4_0:  155 tensors
llama_model_loader: - type q6_K:    1 tensors
llm_load_print_meta: format           = GGUF V2 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = SPM
llm_load_print_meta: n_vocab          = 32003
llm_load_print_meta: n_merges         = 0
llm_load_print_meta: n_ctx_train      = 2048
llm_load_print_meta: n_embd           = 2048
llm_load_print_meta: n_head           = 32
llm_load_print_meta: n_head_kv        = 4
llm_load_print_meta: n_layer          = 22
llm_load_print_meta: n_rot            = 64
llm_load_print_meta: n_gqa            = 8
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: n_ff             = 5632
llm_load_print_meta: freq_base_train  = 10000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: model type       = ?B
llm_load_print_meta: model ftype      = mostly Q4_0
llm_load_print_meta: model params     = 1.10 B
llm_load_print_meta: model size       = 606.54 MiB (4.63 BPW)
llm_load_print_meta: general.name   = py007_tinyllama-1.1b-chat-v0.3
llm_load_print_meta: BOS token = 1 '<s>'
llm_load_print_meta: EOS token = 2 '</s>'
llm_load_print_meta: UNK token = 0 '<unk>'
llm_load_print_meta: LF token  = 13 '<0x0A>'
llm_load_tensors: ggml ctx size =    0.07 MB
llm_load_tensors: using OpenCL for GPU acceleration
llm_load_tensors: mem required  =  606.60 MB
llm_load_tensors: offloading 0 repeating layers to GPU
llm_load_tensors: offloaded 0/23 layers to GPU
llm_load_tensors: VRAM used: 0.00 MB
.......................................................................................
llama_new_context_with_model: n_ctx      = 2048
llama_new_context_with_model: freq_base  = 10000.0
llama_new_context_with_model: freq_scale = 1
llama_new_context_with_model: kv self size  =   44.00 MB
llama_new_context_with_model: compute buffer total size = 150.13 MB

Thread 1 "main" received signal SIGILL, Illegal instruction.
0x000000555561ccf0 in ggml_vec_dot_q4_0_q8_0 (n=2048,
    s=0x7ffffe9a78, vx=0x7cfd9a57c0, vy=0x7f4604bf30)
    at ggml.c:2493
2493            const int32x4_t p_0 = vdotq_s32(vdotq_s32(vdupq_n_s32(0), v0_0ls, v1_0l), v0_0hs, v1_0h);
(gdb) bt
#0  0x000000555561ccf0 in ggml_vec_dot_q4_0_q8_0 (n=2048,
    s=0x7ffffe9a78, vx=0x7cfd9a57c0, vy=0x7f4604bf30)
    at ggml.c:2493
#1  0x0000005555622e50 in ggml_compute_forward_mul_mat (
    params=0x7ffffe9d60, src0=0x7fb6801740, src1=0x7cf8004900,
    dst=0x7cf8004a60) at ggml.c:11991
#2  0x00000055556209e0 in ggml_compute_forward (
    params=0x7ffffe9d60, tensor=0x7cf8004a60) at ggml.c:16705
#3  0x0000005555610320 in ggml_graph_compute_thread (
    data=0x7ffffe9dd0) at ggml.c:18327
#4  0x000000555560ff4c in ggml_graph_compute (
    cgraph=0x7cf7f64060, cplan=0x7ffffe9ed0) at ggml.c:18903
#5  0x000000555564f2a0 in ggml_graph_compute_helper (buf=...,
    graph=0x7cf7f64060, n_threads=1) at llama.cpp:567
#6  0x000000555565004c in llama_decode_internal (lctx=...,
    batch=...) at llama.cpp:5868
#7  0x000000555565074c in llama_decode (ctx=0x7f16040380,
    batch=...) at llama.cpp:9351
#8  0x00000055556e6edc in llama_init_from_gpt_params (
    params=...) at common/common.cpp:867
#9  0x00000055555de938 in main (argc=27, argv=0x7fffffebe8)
    at examples/main/main.cpp:182
(gdb) ls
Undefined command: "ls".  Try "help".
(gdb) l
2488            const int8x16_t v1_1l = vld1q_s8(y1->qs);
2489            const int8x16_t v1_1h = vld1q_s8(y1->qs + 16);
2490
2491    #if defined(__ARM_FEATURE_DOTPROD)
2492            // dot product into int32x4_t
2493            const int32x4_t p_0 = vdotq_s32(vdotq_s32(vdupq_n_s32(0), v0_0ls, v1_0l), v0_0hs, v1_0h);
2494            const int32x4_t p_1 = vdotq_s32(vdotq_s32(vdupq_n_s32(0), v0_1ls, v1_1l), v0_1hs, v1_1h);
2495
2496            sumv0 = vmlaq_n_f32(sumv0, vcvtq_f32_s32(p_0), GGML_FP16_TO_FP32(x0->d)*GGML_FP16_TO_FP32(y0->d));
2497            sumv1 = vmlaq_n_f32(sumv1, vcvtq_f32_s32(p_1), GGML_FP16_TO_FP32(x1->d)*GGML_FP16_TO_FP32(y1->d));
(gdb) q
A debugging session is active.

        Inferior 1 [process 8186] will be killed.

Quit anyway? (y or n) y

I tried another model and the same thing happens but in another part.

➜  llama.cpp git:(master) ✗ gdb ./main            (10-14 11:16)
GNU gdb (GDB) 13.2
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-linux-android".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./main...
(gdb) r -m /data/data/com.termux/files/home/llama.cpp/models/tinyllama-1.1b-chat-v0.3.Q2_K.gguf -f ./prompts/tinyllama-chat.txt -r "<|im_start|>assistant" -r "<|im_start|>" -r "<|im_end|>" --in-suffix "<|im_end|>" --color -c 2048 -ins --temp 0.7 --repeat_penalty 1.1 -t 1 -n -1 -s -1
Starting program: /data/data/com.termux/files/home/llama.cpp/main -m /data/data/com.termux/files/home/llama.cpp/models/tinyllama-1.1b-chat-v0.3.Q2_K.gguf -f ./prompts/tinyllama-chat.txt -r "<|im_start|>assistant" -r "<|im_start|>" -r "<|im_end|>" --in-suffix "<|im_end|>" --color -c 2048 -ins --temp 0.7 --repeat_penalty 1.1 -t 1 -n -1 -s -1
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/data/data/com.termux/files/usr/lib/libthread_db.so".
Log start
main: build = 1381 (11dc109)
main: built with clang version 17.0.2 for aarch64-unknown-linux-android24
main: seed  = 1697293119
ggml_opencl: selecting platform: 'QUALCOMM Snapdragon(TM)'
ggml_opencl: selecting device: 'QUALCOMM Adreno(TM)'
ggml_opencl: device FP16 support: true
[New Thread 0x219e (LWP 8606)]
[New Thread 0x219f (LWP 8607)]
llama_model_loader: loaded meta data with 20 key-value pairs and 201 tensors from /data/data/com.termux/files/home/llama.cpp/models/tinyllama-1.1b-chat-v0.3.Q2_K.gguf (version GGUF V2 (latest))
llama_model_loader: - tensor    0:                token_embd.weight q2_K     [  2048, 32003,     1,     1 ]
llama_model_loader: - tensor    1:              blk.0.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor    2:              blk.0.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor    3:              blk.0.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor    4:         blk.0.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor    5:            blk.0.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor    6:              blk.0.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor    7:            blk.0.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor    8:           blk.0.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor    9:            blk.0.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   10:              blk.1.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   11:              blk.1.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   12:              blk.1.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   13:         blk.1.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   14:            blk.1.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   15:              blk.1.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   16:            blk.1.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   17:           blk.1.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   18:            blk.1.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   19:              blk.2.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   20:              blk.2.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   21:              blk.2.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   22:         blk.2.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   23:            blk.2.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   24:              blk.2.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   25:            blk.2.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   26:           blk.2.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   27:            blk.2.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   28:              blk.3.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   29:              blk.3.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   30:              blk.3.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   31:         blk.3.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   32:            blk.3.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   33:              blk.3.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   34:            blk.3.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   35:           blk.3.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   36:            blk.3.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   37:              blk.4.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   38:              blk.4.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   39:              blk.4.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   40:         blk.4.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   41:            blk.4.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   42:              blk.4.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   43:            blk.4.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   44:           blk.4.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   45:            blk.4.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   46:              blk.5.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   47:              blk.5.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   48:              blk.5.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   49:         blk.5.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   50:            blk.5.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   51:              blk.5.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   52:            blk.5.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   53:           blk.5.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   54:            blk.5.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   55:              blk.6.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   56:              blk.6.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   57:              blk.6.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   58:         blk.6.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   59:            blk.6.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   60:              blk.6.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   61:            blk.6.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   62:           blk.6.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   63:            blk.6.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   64:              blk.7.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   65:              blk.7.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   66:              blk.7.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   67:         blk.7.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   68:            blk.7.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   69:              blk.7.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   70:            blk.7.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   71:           blk.7.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   72:            blk.7.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   73:              blk.8.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   74:              blk.8.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   75:              blk.8.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   76:         blk.8.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   77:            blk.8.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   78:              blk.8.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   79:            blk.8.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   80:           blk.8.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   81:            blk.8.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   82:              blk.9.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   83:              blk.9.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   84:              blk.9.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   85:         blk.9.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   86:            blk.9.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   87:              blk.9.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   88:            blk.9.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   89:           blk.9.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   90:            blk.9.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   91:             blk.10.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   92:             blk.10.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   93:             blk.10.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   94:        blk.10.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   95:           blk.10.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   96:             blk.10.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   97:           blk.10.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   98:          blk.10.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   99:           blk.10.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  100:             blk.11.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  101:             blk.11.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  102:             blk.11.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  103:        blk.11.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  104:           blk.11.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  105:             blk.11.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  106:           blk.11.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  107:          blk.11.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  108:           blk.11.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  109:             blk.12.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  110:             blk.12.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  111:             blk.12.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  112:        blk.12.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  113:           blk.12.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  114:             blk.12.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  115:           blk.12.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  116:          blk.12.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  117:           blk.12.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  118:             blk.13.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  119:             blk.13.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  120:             blk.13.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  121:        blk.13.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  122:           blk.13.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  123:             blk.13.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  124:           blk.13.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  125:          blk.13.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  126:           blk.13.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  127:             blk.14.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  128:             blk.14.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  129:             blk.14.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  130:        blk.14.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  131:           blk.14.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  132:             blk.14.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  133:           blk.14.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  134:          blk.14.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  135:           blk.14.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  136:             blk.15.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  137:             blk.15.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  138:             blk.15.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  139:        blk.15.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  140:           blk.15.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  141:             blk.15.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  142:           blk.15.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  143:          blk.15.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  144:           blk.15.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  145:             blk.16.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  146:             blk.16.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  147:             blk.16.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  148:        blk.16.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  149:           blk.16.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  150:             blk.16.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  151:           blk.16.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  152:          blk.16.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  153:           blk.16.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  154:             blk.17.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  155:             blk.17.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  156:             blk.17.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  157:        blk.17.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  158:           blk.17.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  159:             blk.17.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  160:           blk.17.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  161:          blk.17.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  162:           blk.17.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  163:             blk.18.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  164:             blk.18.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  165:             blk.18.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  166:        blk.18.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  167:           blk.18.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  168:             blk.18.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  169:           blk.18.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  170:          blk.18.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  171:           blk.18.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  172:             blk.19.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  173:             blk.19.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  174:             blk.19.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  175:        blk.19.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  176:           blk.19.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  177:             blk.19.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  178:           blk.19.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  179:          blk.19.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  180:           blk.19.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  181:             blk.20.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  182:             blk.20.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  183:             blk.20.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  184:        blk.20.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  185:           blk.20.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  186:             blk.20.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  187:           blk.20.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  188:          blk.20.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  189:           blk.20.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  190:             blk.21.attn_q.weight q2_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  191:             blk.21.attn_k.weight q2_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  192:             blk.21.attn_v.weight q3_K     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  193:        blk.21.attn_output.weight q3_K     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  194:           blk.21.ffn_gate.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  195:             blk.21.ffn_up.weight q3_K     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  196:           blk.21.ffn_down.weight q3_K     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  197:          blk.21.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  198:           blk.21.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  199:               output_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  200:                    output.weight q6_K     [  2048, 32003,     1,     1 ]
llama_model_loader: - kv   0:                       general.architecture str
llama_model_loader: - kv   1:                               general.name str
llama_model_loader: - kv   2:                       llama.context_length u32
llama_model_loader: - kv   3:                     llama.embedding_length u32
llama_model_loader: - kv   4:                          llama.block_count u32
llama_model_loader: - kv   5:                  llama.feed_forward_length u32
llama_model_loader: - kv   6:                 llama.rope.dimension_count u32
llama_model_loader: - kv   7:                 llama.attention.head_count u32
llama_model_loader: - kv   8:              llama.attention.head_count_kv u32
llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32
llama_model_loader: - kv  10:                       llama.rope.freq_base f32
llama_model_loader: - kv  11:                          general.file_type u32
llama_model_loader: - kv  12:                       tokenizer.ggml.model str
llama_model_loader: - kv  13:                      tokenizer.ggml.tokens arr
llama_model_loader: - kv  14:                      tokenizer.ggml.scores arr
llama_model_loader: - kv  15:                  tokenizer.ggml.token_type arr
llama_model_loader: - kv  16:                tokenizer.ggml.bos_token_id u32
llama_model_loader: - kv  17:                tokenizer.ggml.eos_token_id u32
llama_model_loader: - kv  18:            tokenizer.ggml.unknown_token_id u32
llama_model_loader: - kv  19:               general.quantization_version u32
llama_model_loader: - type  f32:   45 tensors
llama_model_loader: - type q2_K:   45 tensors
llama_model_loader: - type q3_K:  110 tensors
llama_model_loader: - type q6_K:    1 tensors
llm_load_print_meta: format           = GGUF V2 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = SPM
llm_load_print_meta: n_vocab          = 32003
llm_load_print_meta: n_merges         = 0
llm_load_print_meta: n_ctx_train      = 2048
llm_load_print_meta: n_embd           = 2048
llm_load_print_meta: n_head           = 32
llm_load_print_meta: n_head_kv        = 4
llm_load_print_meta: n_layer          = 22
llm_load_print_meta: n_rot            = 64
llm_load_print_meta: n_gqa            = 8
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: n_ff             = 5632
llm_load_print_meta: freq_base_train  = 10000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: model type       = ?B
llm_load_print_meta: model ftype      = mostly Q2_K
llm_load_print_meta: model params     = 1.10 B
llm_load_print_meta: model size       = 459.11 MiB (3.50 BPW)
llm_load_print_meta: general.name   = py007_tinyllama-1.1b-chat-v0.3
llm_load_print_meta: BOS token = 1 '<s>'
llm_load_print_meta: EOS token = 2 '</s>'
llm_load_print_meta: UNK token = 0 '<unk>'
llm_load_print_meta: LF token  = 13 '<0x0A>'
llm_load_tensors: ggml ctx size =    0.07 MB
llm_load_tensors: using OpenCL for GPU acceleration
llm_load_tensors: mem required  =  459.18 MB
llm_load_tensors: offloading 0 repeating layers to GPU
llm_load_tensors: offloaded 0/23 layers to GPU
llm_load_tensors: VRAM used: 0.00 MB
......................................................................................
llama_new_context_with_model: n_ctx      = 2048
llama_new_context_with_model: freq_base  = 10000.0
llama_new_context_with_model: freq_scale = 1
llama_new_context_with_model: kv self size  =   44.00 MB
llama_new_context_with_model: compute buffer total size = 150.13 MB

Thread 1 "main" received signal SIGILL, Illegal instruction.
0x000000555572402c in ggml_vec_dot_q2_K_q8_K (n=2048,
    s=0x7ffffe9a78, vx=0x7d05d2a220, vy=0x7f46032f30)
    at k_quants.c:1399
1399                MULTIPLY_ACCUM_WITH_SCALE(0);
(gdb) bt
#0  0x000000555572402c in ggml_vec_dot_q2_K_q8_K (n=2048,
    s=0x7ffffe9a78, vx=0x7d05d2a220, vy=0x7f46032f30)
    at k_quants.c:1399
#1  0x0000005555622e50 in ggml_compute_forward_mul_mat (
    params=0x7ffffe9d60, src0=0x7fb66f3740, src1=0x7d0131f900,
    dst=0x7d0131fa60) at ggml.c:11991
#2  0x00000055556209e0 in ggml_compute_forward (
    params=0x7ffffe9d60, tensor=0x7d0131fa60) at ggml.c:16705
#3  0x0000005555610320 in ggml_graph_compute_thread (
    data=0x7ffffe9dd0) at ggml.c:18327
#4  0x000000555560ff4c in ggml_graph_compute (
    cgraph=0x7d0127f060, cplan=0x7ffffe9ed0) at ggml.c:18903
#5  0x000000555564f2a0 in ggml_graph_compute_helper (buf=...,
    graph=0x7d0127f060, n_threads=1) at llama.cpp:567
#6  0x000000555565004c in llama_decode_internal (lctx=...,
    batch=...) at llama.cpp:5868
#7  0x000000555565074c in llama_decode (ctx=0x7f1602ebe0,
    batch=...) at llama.cpp:9351
#8  0x00000055556e6edc in llama_init_from_gpt_params (
    params=...) at common/common.cpp:867
#9  0x00000055555de938 in main (argc=27, argv=0x7fffffebe8)
    at examples/main/main.cpp:182
(gdb) l
1394                const uint8x16x2_t q2bits = vld1q_u8_x2(q2); q2 += 32;
1395
1396                int8x16x2_t q8bytes = vld1q_s8_x2(q8); q8 += 32;
1397                q2bytes.val[0] = vreinterpretq_s8_u8(vandq_u8(q2bits.val[0], m3));
1398                q2bytes.val[1] = vreinterpretq_s8_u8(vandq_u8(q2bits.val[1], m3));
1399                MULTIPLY_ACCUM_WITH_SCALE(0);
1400
1401                SHIFT_MULTIPLY_ACCUM_WITH_SCALE(2, 2);
1402
1403                SHIFT_MULTIPLY_ACCUM_WITH_SCALE(4, 4);
(gdb) q
A debugging session is active.

        Inferior 1 [process 8604] will be killed.

Quit anyway? (y or n) y

➜  llama.cpp git:(master) ✗ git pull              (10-14 11:33)
Already up to date.

Thank you in advance for the help, greetings.

staviq commented 12 months ago

@ElvisClaros Hmm... I cannot reproduce, it works fine for me. The only thing that comes to my mind is that CPU capabilities are incorrectly detected by the compiler in your case.

Can you try editing Makefile and commenting out or removing those lines, and compiling again ?

https://github.com/ggerganov/llama.cpp/blob/11dc1091f64b24ca6d643acc6d0051117ba60161/Makefile#L310-L311

ElvisClaros commented 12 months ago

Hi @staviq. Thank you very much, it already works.

➜  llama.cpp git:(master) ✗ nv Makefile           (10-14 13:40)

➜  llama.cpp git:(master) ✗ make clean            (10-14 13:40)
I llama.cpp build info:                                         I UNAME_S:   Linux
I UNAME_P:   unknown                                            I UNAME_M:   aarch64                                            I CFLAGS:    -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c11   -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -pthread            I CXXFLAGS:  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi
I NVCCFLAGS:  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread    -Wno-pedantic -Xcompiler "-Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi "
I LDFLAGS:
I CC:        clang version 17.0.2
I CXX:       clang version 17.0.2

rm -vrf *.o tests/*.o *.so *.dll benchmark-matmult build-info.h *.dot *.gcno tests/*.gcno *.gcda tests/*.gcda *.gcov tests/*.gcov lcov-report gcovr-report main quantize quantize-stats perplexity embedding vdot q8dot train-text-from-scratch convert-llama2c-to-ggml simple batched batched-bench save-load-state server embd-input-test gguf llama-bench llava baby-llama beam-search speculative infill benchmark-matmult parallel finetune export-lora tests/test-c.o metal tests/test-llama-grammar tests/test-grammar-parser tests/test-double-float tests/test-grad0 tests/test-opt tests/test-quantize-fns tests/test-quantize-perf tests/test-sampling tests/test-tokenizer-0-llama tests/test-tokenizer-0-falcon tests/test-tokenizer-1-llama tests/test-tokenizer-1-bpe
removed 'common.o'
removed 'console.o'
removed 'ggml-alloc.o'
removed 'ggml-backend.o'
removed 'ggml-opencl.o'
removed 'ggml.o'
removed 'grammar-parser.o'
removed 'k_quants.o'
removed 'llama.o'
removed 'sampling.o'
removed 'train.o'
removed 'tests/test-c.o'
removed 'libembdinput.so'
removed 'benchmark-matmult'
removed 'build-info.h'
removed 'main'
removed 'quantize'
removed 'quantize-stats'
removed 'perplexity'
removed 'embedding'
removed 'vdot'
removed 'q8dot'
removed 'train-text-from-scratch'
removed 'convert-llama2c-to-ggml'
removed 'simple'
removed 'batched'
removed 'batched-bench'
removed 'save-load-state'
removed 'server'
removed 'embd-input-test'
removed 'gguf'
removed 'llama-bench'
removed 'llava'
removed 'baby-llama'
removed 'beam-search'
removed 'speculative'
removed 'infill'
removed 'parallel'
removed 'finetune'
removed 'export-lora'

➜  llama.cpp git:(master) ✗ make LLAMA_CLBLAST=1 LLAMA_DEBUG=1 -j8
I llama.cpp build info:
I UNAME_S:   Linux
I UNAME_P:   unknown
I UNAME_M:   aarch64
I CFLAGS:    -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c11   -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -pthread
I CXXFLAGS:  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi
I NVCCFLAGS:  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread     -Wno-pedantic -Xcompiler "-Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi "
I LDFLAGS:   -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
I CC:        clang version 17.0.2
I CXX:       clang version 17.0.2

cc  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c11   -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -pthread     -c ggml.c -o ggml.o
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  -c llama.cpp -o llama.o
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  -c common/console.cpp -o console.o
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  -c common/grammar-parser.cpp -o grammar-parser.o
cc -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c11   -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -pthread   -c k_quants.c -o k_quants.o
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  -c ggml-opencl.cpp -o ggml-opencl.o
cc  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c11   -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -pthread     -c ggml-alloc.c -o ggml-alloc.o
cc  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c11   -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -pthread     -c ggml-backend.c -o ggml-backend.o
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  -c common/train.cpp -o train.o
cc -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c11   -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wunreachable-code-break -Wunreachable-code-return -Wdouble-promotion -pthread   -c tests/test-c.c -o tests/test-c.o
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  -c common/common.cpp -o common.o
ggml.c:2432:5: warning: implicit conversion increases floating-point precision: 'float32_t' (aka 'float') to 'ggml_float' (aka 'double') [-Wdouble-promotion]
 2432 |     GGML_F16_VEC_REDUCE(sumf, sum);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:1959:41: note: expanded from macro 'GGML_F16_VEC_REDUCE'
 1959 |     #define GGML_F16_VEC_REDUCE         GGML_F32Cx4_REDUCE
      |                                         ^
ggml.c:1949:38: note: expanded from macro 'GGML_F32Cx4_REDUCE'
 1949 |     #define GGML_F32Cx4_REDUCE       GGML_F32x4_REDUCE
      |                                      ^
ggml.c:1879:11: note: expanded from macro 'GGML_F32x4_REDUCE'
 1879 |     res = GGML_F32x4_REDUCE_ONE(x[0]);         \
      |         ~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:1864:34: note: expanded from macro 'GGML_F32x4_REDUCE_ONE'
 1864 | #define GGML_F32x4_REDUCE_ONE(x) vaddvq_f32(x)
      |                                  ^~~~~~~~~~~~~
ggml.c:3692:9: warning: implicit conversion increases floating-point precision: 'float32_t' (aka 'float') to 'ggml_float' (aka 'double') [-Wdouble-promotion]
 3692 |         GGML_F16_VEC_REDUCE(sumf[k], sum[k]);
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:1959:41: note: expanded from macro 'GGML_F16_VEC_REDUCE'
 1959 |     #define GGML_F16_VEC_REDUCE         GGML_F32Cx4_REDUCE
      |                                         ^
ggml.c:1949:38: note: expanded from macro 'GGML_F32Cx4_REDUCE'
 1949 |     #define GGML_F32Cx4_REDUCE       GGML_F32x4_REDUCE
      |                                      ^
ggml.c:1879:11: note: expanded from macro 'GGML_F32x4_REDUCE'
 1879 |     res = GGML_F32x4_REDUCE_ONE(x[0]);         \
      |         ~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~
ggml.c:1864:34: note: expanded from macro 'GGML_F32x4_REDUCE_ONE'
 1864 | #define GGML_F32x4_REDUCE_ONE(x) vaddvq_f32(x)
      |                                  ^~~~~~~~~~~~~
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  -c common/sampling.cpp -o sampling.o
2 warnings generated.
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  pocs/vdot/vdot.cpp ggml.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o vdot -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  pocs/vdot/q8dot.cpp ggml.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o q8dot -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/benchmark/benchmark-matmult.cpp ggml.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o benchmark-matmult -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/main/main.cpp ggml.o llama.o common.o sampling.o console.o grammar-parser.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o main -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/quantize/quantize.cpp ggml.o llama.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o quantize -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/quantize-stats/quantize-stats.cpp ggml.o llama.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o quantize-stats -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/perplexity/perplexity.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o perplexity -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/embedding/embedding.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o embedding -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/train-text-from-scratch/train-text-from-scratch.cpp ggml.o llama.o common.o sampling.o train.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o train-text-from-scratch -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp ggml.o llama.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o convert-llama2c-to-ggml -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/simple/simple.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o simple -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/batched/batched.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o batched -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/batched-bench/batched-bench.cpp ggml.o llama.o common.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o batched-bench -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/save-load-state/save-load-state.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o save-load-state -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  -Iexamples/server examples/server/server.cpp ggml.o llama.o common.o sampling.o grammar-parser.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o server -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ --shared -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/embd-input/embd-input-lib.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o libembdinput.so -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/gguf/gguf.cpp ggml.o llama.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o gguf -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL

====  Run ./main -h for help.  ====

aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/llama-bench/llama-bench.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o llama-bench -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/llava/llava.cpp examples/llava/clip.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o llava -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL  -Wno-cast-qual
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/baby-llama/baby-llama.cpp ggml.o llama.o common.o sampling.o train.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o baby-llama -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/beam-search/beam-search.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o beam-search -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/speculative/speculative.cpp ggml.o llama.o common.o sampling.o grammar-parser.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o speculative -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/infill/infill.cpp ggml.o llama.o common.o sampling.o console.o grammar-parser.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o infill -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/parallel/parallel.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o parallel -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/finetune/finetune.cpp ggml.o llama.o common.o sampling.o train.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o finetune -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/export-lora/export-lora.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o export-lora -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL
aarch64-linux-android-clang++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DGGML_USE_K_QUANTS -DGGML_USE_CLBLAST -I/data/data/com.termux/files/usr/include  -std=c++11 -fPIC -O3 -O0 -g -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread   -Wunreachable-code-break -Wunreachable-code-return -Wmissing-prototypes -Wextra-semi  examples/embd-input/embd-input-test.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-opencl.o ggml-alloc.o ggml-backend.o -o embd-input-test -g -L/data/data/com.termux/files/usr//data/data/com.termux/files/usr/lib -L/data/data/com.termux/files/usr/lib -lclblast -lOpenCL  -L. -lembdinput

➜  llama.cpp git:(master) ✗ GGML_OPENCL_PLATFORM=0              GGML_OPENCL_DEVICE=0
export LD_LIBRARY_PATH=/vendor/lib64:$LD_LIBRARY_PATH

➜  llama.cpp git:(master) ✗ ./main -m /data/data/com.termux/files/home/llama.cpp/models/tinyllama-1.1b-chat-v0.3.Q4_0.gguf -f ./prompts/tinyllama-chat.txt -r "<|im_start|>assistant" -r "<|im_start|>" -r "<|im_end|>" --in-suffix "<|im_end|>" --color -c 2048 -ins --temp 0.7 --repeat_penalty 1.1 -t 8 -n -1 -s -1
Log start
main: build = 1381 (11dc109)
main: built with clang version 17.0.2 for aarch64-unknown-linux-android24
main: seed  = 1697301926
ggml_opencl: selecting platform: 'QUALCOMM Snapdragon(TM)'
ggml_opencl: selecting device: 'QUALCOMM Adreno(TM)'
ggml_opencl: device FP16 support: true
llama_model_loader: loaded meta data with 20 key-value pairs and 201 tensors from /data/data/com.termux/files/home/llama.cpp/models/tinyllama-1.1b-chat-v0.3.Q4_0.gguf (version GGUF V2 (latest))
llama_model_loader: - tensor    0:                token_embd.weight q4_0     [  2048, 32003,     1,     1 ]
llama_model_loader: - tensor    1:              blk.0.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor    2:              blk.0.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor    3:              blk.0.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor    4:         blk.0.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor    5:            blk.0.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor    6:              blk.0.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor    7:            blk.0.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor    8:           blk.0.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor    9:            blk.0.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   10:              blk.1.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   11:              blk.1.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   12:              blk.1.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   13:         blk.1.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   14:            blk.1.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   15:              blk.1.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   16:            blk.1.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   17:           blk.1.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   18:            blk.1.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   19:              blk.2.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   20:              blk.2.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   21:              blk.2.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   22:         blk.2.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   23:            blk.2.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   24:              blk.2.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   25:            blk.2.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   26:           blk.2.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   27:            blk.2.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   28:              blk.3.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   29:              blk.3.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   30:              blk.3.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   31:         blk.3.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   32:            blk.3.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   33:              blk.3.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   34:            blk.3.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   35:           blk.3.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   36:            blk.3.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   37:              blk.4.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   38:              blk.4.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   39:              blk.4.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   40:         blk.4.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   41:            blk.4.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   42:              blk.4.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   43:            blk.4.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   44:           blk.4.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   45:            blk.4.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   46:              blk.5.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   47:              blk.5.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   48:              blk.5.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   49:         blk.5.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   50:            blk.5.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   51:              blk.5.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   52:            blk.5.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   53:           blk.5.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   54:            blk.5.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   55:              blk.6.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   56:              blk.6.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   57:              blk.6.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   58:         blk.6.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   59:            blk.6.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   60:              blk.6.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   61:            blk.6.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   62:           blk.6.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   63:            blk.6.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   64:              blk.7.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   65:              blk.7.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   66:              blk.7.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   67:         blk.7.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   68:            blk.7.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   69:              blk.7.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   70:            blk.7.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   71:           blk.7.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   72:            blk.7.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   73:              blk.8.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   74:              blk.8.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   75:              blk.8.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   76:         blk.8.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   77:            blk.8.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   78:              blk.8.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   79:            blk.8.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   80:           blk.8.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   81:            blk.8.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   82:              blk.9.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   83:              blk.9.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   84:              blk.9.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   85:         blk.9.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   86:            blk.9.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   87:              blk.9.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   88:            blk.9.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   89:           blk.9.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   90:            blk.9.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   91:             blk.10.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   92:             blk.10.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   93:             blk.10.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor   94:        blk.10.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor   95:           blk.10.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   96:             blk.10.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor   97:           blk.10.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor   98:          blk.10.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor   99:           blk.10.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  100:             blk.11.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  101:             blk.11.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  102:             blk.11.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  103:        blk.11.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  104:           blk.11.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  105:             blk.11.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  106:           blk.11.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  107:          blk.11.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  108:           blk.11.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  109:             blk.12.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  110:             blk.12.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  111:             blk.12.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  112:        blk.12.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  113:           blk.12.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  114:             blk.12.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  115:           blk.12.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  116:          blk.12.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  117:           blk.12.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  118:             blk.13.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  119:             blk.13.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  120:             blk.13.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  121:        blk.13.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  122:           blk.13.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  123:             blk.13.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  124:           blk.13.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  125:          blk.13.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  126:           blk.13.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  127:             blk.14.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  128:             blk.14.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  129:             blk.14.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  130:        blk.14.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  131:           blk.14.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  132:             blk.14.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  133:           blk.14.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  134:          blk.14.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  135:           blk.14.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  136:             blk.15.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  137:             blk.15.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  138:             blk.15.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  139:        blk.15.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  140:           blk.15.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  141:             blk.15.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  142:           blk.15.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  143:          blk.15.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  144:           blk.15.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  145:             blk.16.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  146:             blk.16.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  147:             blk.16.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  148:        blk.16.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  149:           blk.16.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  150:             blk.16.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  151:           blk.16.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  152:          blk.16.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  153:           blk.16.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  154:             blk.17.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  155:             blk.17.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  156:             blk.17.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  157:        blk.17.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  158:           blk.17.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  159:             blk.17.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  160:           blk.17.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  161:          blk.17.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  162:           blk.17.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  163:             blk.18.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  164:             blk.18.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  165:             blk.18.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  166:        blk.18.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  167:           blk.18.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  168:             blk.18.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  169:           blk.18.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  170:          blk.18.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  171:           blk.18.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  172:             blk.19.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  173:             blk.19.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  174:             blk.19.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  175:        blk.19.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  176:           blk.19.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  177:             blk.19.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  178:           blk.19.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  179:          blk.19.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  180:           blk.19.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  181:             blk.20.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  182:             blk.20.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  183:             blk.20.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  184:        blk.20.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  185:           blk.20.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  186:             blk.20.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  187:           blk.20.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  188:          blk.20.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  189:           blk.20.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  190:             blk.21.attn_q.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  191:             blk.21.attn_k.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  192:             blk.21.attn_v.weight q4_0     [  2048,   256,     1,     1 ]
llama_model_loader: - tensor  193:        blk.21.attn_output.weight q4_0     [  2048,  2048,     1,     1 ]
llama_model_loader: - tensor  194:           blk.21.ffn_gate.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  195:             blk.21.ffn_up.weight q4_0     [  2048,  5632,     1,     1 ]
llama_model_loader: - tensor  196:           blk.21.ffn_down.weight q4_0     [  5632,  2048,     1,     1 ]
llama_model_loader: - tensor  197:          blk.21.attn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  198:           blk.21.ffn_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  199:               output_norm.weight f32      [  2048,     1,     1,     1 ]
llama_model_loader: - tensor  200:                    output.weight q6_K     [  2048, 32003,     1,     1 ]
llama_model_loader: - kv   0:                       general.architecture str
llama_model_loader: - kv   1:                               general.name str
llama_model_loader: - kv   2:                       llama.context_length u32
llama_model_loader: - kv   3:                     llama.embedding_length u32
llama_model_loader: - kv   4:                          llama.block_count u32
llama_model_loader: - kv   5:                  llama.feed_forward_length u32
llama_model_loader: - kv   6:                 llama.rope.dimension_count u32
llama_model_loader: - kv   7:                 llama.attention.head_count u32
llama_model_loader: - kv   8:              llama.attention.head_count_kv u32
llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32
llama_model_loader: - kv  10:                       llama.rope.freq_base f32
llama_model_loader: - kv  11:                          general.file_type u32
llama_model_loader: - kv  12:                       tokenizer.ggml.model str
llama_model_loader: - kv  13:                      tokenizer.ggml.tokens arr
llama_model_loader: - kv  14:                      tokenizer.ggml.scores arr
llama_model_loader: - kv  15:                  tokenizer.ggml.token_type arr
llama_model_loader: - kv  16:                tokenizer.ggml.bos_token_id u32
llama_model_loader: - kv  17:                tokenizer.ggml.eos_token_id u32
llama_model_loader: - kv  18:            tokenizer.ggml.unknown_token_id u32
llama_model_loader: - kv  19:               general.quantization_version u32
llama_model_loader: - type  f32:   45 tensors
llama_model_loader: - type q4_0:  155 tensors
llama_model_loader: - type q6_K:    1 tensors
llm_load_print_meta: format           = GGUF V2 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = SPM
llm_load_print_meta: n_vocab          = 32003
llm_load_print_meta: n_merges         = 0
llm_load_print_meta: n_ctx_train      = 2048
llm_load_print_meta: n_embd           = 2048
llm_load_print_meta: n_head           = 32
llm_load_print_meta: n_head_kv        = 4
llm_load_print_meta: n_layer          = 22
llm_load_print_meta: n_rot            = 64
llm_load_print_meta: n_gqa            = 8
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: n_ff             = 5632
llm_load_print_meta: freq_base_train  = 10000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: model type       = ?B
llm_load_print_meta: model ftype      = mostly Q4_0
llm_load_print_meta: model params     = 1.10 B
llm_load_print_meta: model size       = 606.54 MiB (4.63 BPW)
llm_load_print_meta: general.name   = py007_tinyllama-1.1b-chat-v0.3
llm_load_print_meta: BOS token = 1 '<s>'
llm_load_print_meta: EOS token = 2 '</s>'
llm_load_print_meta: UNK token = 0 '<unk>'
llm_load_print_meta: LF token  = 13 '<0x0A>'
llm_load_tensors: ggml ctx size =    0.07 MB
llm_load_tensors: using OpenCL for GPU acceleration
llm_load_tensors: mem required  =  606.60 MB
llm_load_tensors: offloading 0 repeating layers to GPU
llm_load_tensors: offloaded 0/23 layers to GPU
llm_load_tensors: VRAM used: 0.00 MB
.......................................................................................
llama_new_context_with_model: n_ctx      = 2048
llama_new_context_with_model: freq_base  = 10000.0
llama_new_context_with_model: freq_scale = 1
llama_new_context_with_model: kv self size  =   44.00 MB
llama_new_context_with_model: compute buffer total size = 150.13 MB

system_info: n_threads = 8 / 8 | AVX = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 |
main: interactive mode on.
Reverse prompt: '<|im_start|>assistant'
Reverse prompt: '<|im_start|>'
Reverse prompt: '<|im_end|>'
Reverse prompt: '### Instruction:

'
Input suffix: '<|im_end|>'
sampling: repeat_last_n = 64, repeat_penalty = 1.100000, presence_penalty = 0.000000, frequency_penalty = 0.000000, top_k = 40, tfs_z = 1.000000, top_p = 0.950000, typical_p = 1.000000, temp = 0.700000, mirostat = 0, mirostat_lr = 0.100000, mirostat_ent = 5.000000
generate: n_ctx = 2048, n_batch = 512, n_predict = -1, n_keep = 30

== Running in interactive mode. ==
 - Press Ctrl+C to interject at any time.
 - Press Return to return control to LLaMa.
 - To return control without starting a new line, end your input with '/'.
 - If you want to submit another line, end your input with '\'.

 Below is an instruction that describes a task. Write a response that appropriately completes the request.
<|im_start|>user
> Hola
<|im_end|>Hola Instructor

¿Cómo estás hoy? ¿En qué puedo ayudarte hoy?<|im_end|>
>

llama_print_timings:        load time =    3158.07 ms
llama_print_timings:      sample time =     420.35 ms /    35 runs   (   12.01 ms per token,    83.26 tokens per second)
llama_print_timings: prompt eval time =   66521.82 ms /    60 tokens ( 1108.70 ms per token,     0.90 tokens per second)
llama_print_timings:        eval time =   47764.57 ms /    34 runs   ( 1404.84 ms per token,     0.71 tokens per second)
llama_print_timings:       total time =  184878.31 ms

staviq commented 12 months ago

@ElvisClaros Ok, thank you for confirming.

By the way, that's not complete/correct prompt format, give me a moment and I'll give you correct arguments for main for ChatML format

Also, you might want to beta test #3538 because current master branch ignores those ChatML tags ( you can just git clone --branch specialtokens https://github.com/staviq/llama.cpp.git specialtokens to get that PR in one go )

You can save that -p prompt argument to your prompt file and call it with -f, but you have to replace \n with newlines and make sure that prompt file ends with a newline too, they seem to be important with ChatML format.

You can confirm the prompt format was properly processes by adding --verbose-prompt, this will print tokenized prompt format in the output after model loads.

If everything goes correctly, tokenized prompt should not contain <|im_start|> or <|im_end|> but an empty string with token number assigned.

Please note, main in that PR does process special tokens correctly, but they are still shown on the screen during the chat, you can ignore that as long as --verbose-prompt shows that prompt format tokenized properly.

theoctopusride commented 12 months ago

Can you get a stack trace when it crashes?

@theoctopusride Run it with gdb --args ./build/bin/main -m /sdcard/Download/llama-2-7b-chat.Q3_K_S.gguf -color -c 2048 --keep 1 -t 3 -b 10 -i -ins

When gdb starts type r (enter) to start

When it crashes, type bt (enter) and post the output.

I provided 2 results below using gdb with `./build/bin/main -m /sdcard/Download/llama-2-7b-chat.Q3_K_S.gguf --color -c 2048 --keep 1 -t 3 -b 10 -i -ins':

output for compile with $ cmake .. -DLLAMA_SANITIZE_ADDRESS=ON && cmake --build . --config Debug:

Starting program: /data/data/com.termux/files/home/llama.cpp/build/bin/main -m /sdcard/Download/llama-2-7b-chat.Q3_K_S.gguf --color -c 2048 --keep 1 -t 3 -b 10 -i -ins
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/data/data/com.termux/files/usr/lib/libthread_db.so".
warning: section .note.gnu.build-id not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .dynsym not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .gnu.version not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .gnu.version_d not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .gnu.hash not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .dynstr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .rela.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .eh_frame not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .dynamic not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .got.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .bss not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .note.android.ident not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .dynsym not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .gnu.version not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .gnu.version_d not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .gnu.version_r not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .gnu.hash not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .dynstr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .rela.dyn not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .relr.dyn not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .rela.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .rodata not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .eh_frame not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .data.rel.ro not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .fini_array not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .dynamic not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .got not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .got.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .data not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .bss not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .gnu.version_d not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .got not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .data not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .gcc_except_table not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .gnu.version_d not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .gcc_except_table not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .gnu.version_d not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .got not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .dynsym not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .gnu.version not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .gnu.version_d not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .gnu.hash not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .dynstr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .rela.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .eh_frame not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .dynamic not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .got.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libnetd_client.so
[New Thread 0x6b34 (LWP 27444)]
[New Thread 0x6b35 (LWP 27445)]

Thread 2 "main" received signal SIGILL, Illegal instruction.
[Switching to Thread 0x6b34 (LWP 27444)]
0x0000007ff56bcedc in __interceptor_prctl () at out/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1376
1376    out/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc: No such file or directory.
#0  0x0000007ff56bcedc in __interceptor_prctl () at out/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:1376
#1  0x0000005c5ccc6cf0 [PAC] in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

output for compile with $ cmake .. && cmake --build . --config Debug:

Starting program: /data/data/com.termux/files/home/llama.cpp/build/bin/main -m /sdcard/Download/llama-2-7b-chat.Q3_K_S.gguf --color -c --keep 1 -t 3 -b 10 -i -ins
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/data/data/com.termux/files/usr/lib/libthread_db.so".
warning: section .note.gnu.build-id not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .dynsym not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .gnu.version not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .gnu.version_d not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .gnu.hash not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .dynstr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .rela.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .eh_frame not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .dynamic not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .got.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .bss not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .note.android.ident not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .dynsym not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .gnu.version not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .gnu.version_d not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .gnu.version_r not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .gnu.hash not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .dynstr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .rela.dyn not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .relr.dyn not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .rela.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .rodata not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .eh_frame not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .data.rel.ro not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .fini_array not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .dynamic not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .got not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .got.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .data not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .bss not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .gnu.version_d not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .gcc_except_table not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .gcc_except_table not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .gnu.version_d not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .got not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .data not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .gnu.version_d not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .got not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .dynsym not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .gnu.version not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .gnu.version_d not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .gnu.hash not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .dynstr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .rela.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .eh_frame not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .dynamic not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .got.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libnetd_client.so

Program received signal SIGABRT, Aborted.
0x0000007ff23f17b0 in abort () from /apex/com.android.runtime/lib64/bionic/libc.so
#0  0x0000007ff23f17b0 in abort () from /apex/com.android.runtime/lib64/bionic/libc.so
#1  0x0000007ff6354fd0 in ?? () from /data/data/com.termux/files/usr/lib/libc++_shared.so
#2  0x0000007ff6355184 [PAC] in ?? () from /data/data/com.termux/files/usr/lib/libc++_shared.so
#3  0x0000007ff6369e84 [PAC] in ?? () from /data/data/com.termux/files/usr/lib/libc++_shared.so
#4  0x0000007ff6369518 [PAC] in ?? () from /data/data/com.termux/files/usr/lib/libc++_shared.so
#5  0x0000007ff6369470 [PAC] in __cxa_throw () from /data/data/com.termux/files/usr/lib/libc++_shared.so
#6  0x0000007ff634dfdc [PAC] in ?? () from /data/data/com.termux/files/usr/lib/libc++_shared.so
#7  0x0000007ff634a834 [PAC] in std::__ndk1::stoi(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&, unsigned long*, int) () from /data/data/com.termux/files/usr/lib/libc++_shared.so
#8  0x0000005555595c38 [PAC] in gpt_params_parse (argc=13, argv=0x7fffffed18, params=...) at /data/data/com.termux/files/home/llama.cpp/common/common.cpp:194
#9  0x00000055555889a4 in main (argc=<optimized out>, argv=<optimized out>) at /data/data/com.termux/files/home/llama.cpp/examples/main/main.cpp:109

Thank you for your help

staviq commented 12 months ago

@theoctopusride

--color -c --keep 1

You gave it -c argument with no value, that caused the text --keep to be interpreted as a number which -c expects, which made it crash on exception.

Edit: With the first one, you got corrupt stack, which means something is seriously out of whack with that binary, I would recommend deleting and re downloading llama.cpp and building again from scratch.

theoctopusride commented 12 months ago

ugh, typing it in by hand in termux and messed up the 2nd time. I'll send the output in a little bit with the good cmd

theoctopusride commented 12 months ago

ok re-cloned and compiled using $ cmake .. && cmake --build . --config Debug

ran with gdb using ./build/bin/main -m /sdcard/Download/llama-2-7b-chat.Q3_K_S.gguf --color -c 2048 --keep 1 -t 3 -b 10 -i -ins

Starting program: /data/data/com.termux/files/home/llama.cpp/build/bin/main -m /sdcard/Download/llama-2-7b-chat.Q3_K_S.gguf --color -c 2048 --keep 1 -t 3 -b 10 -i -ins
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/data/data/com.termux/files/usr/lib/libthread_db.so".
...
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so    
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so  
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so  
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so  
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so    
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so                                              
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so  
warning: section .plt not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so       
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so                                               
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so   
warning: section .got not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so       
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so   
warning: section .data not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so      
warning: section .bss not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so       
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libc_secshared.so                                            
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libc_secshared.so                                             
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libc_secshared.so        
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libc_secshared.so   
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libc_secshared.so 
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libc_secshared.so      
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libc_secshared.so        
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libc_secshared.so      
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libc_secshared.so      
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libc_secshared.so      
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libc_secshared.so        
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libc_secshared.so  
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libc_secshared.so      
warning: section .plt not found in .gnu_debugdata for /system/lib64/libc_secshared.so           
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libc_secshared.so   
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libc_secshared.so    
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libc_secshared.so       
warning: section .got not found in .gnu_debugdata for /system/lib64/libc_secshared.so           
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libc_secshared.so       
warning: section .data not found in .gnu_debugdata for /system/lib64/libc_secshared.so          
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libutils.so  
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libutils.so   
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libutils.so              
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libutils.so         
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libutils.so       
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libutils.so            
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libutils.so              
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libutils.so            
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libutils.so            
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libutils.so            
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libutils.so              
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libutils.so        
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libutils.so            
warning: section .plt not found in .gnu_debugdata for /system/lib64/libutils.so                 
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libutils.so         
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libutils.so          
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libutils.so          
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libutils.so             
warning: section .got not found in .gnu_debugdata for /system/lib64/libutils.so                 
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libutils.so             
warning: section .data not found in .gnu_debugdata for /system/lib64/libutils.so                
warning: section .bss not found in .gnu_debugdata for /system/lib64/libutils.so                 
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libunwindstack.so                                            
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libunwindstack.so                                             
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libunwindstack.so        
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libunwindstack.so   
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libunwindstack.so 
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libunwindstack.so      
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libunwindstack.so        
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libunwindstack.so      
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libunwindstack.so      
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libunwindstack.so      
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libunwindstack.so        
warning: section .gcc_except_table not found in .gnu_debugdata for /system/lib64/libunwindstack.so                                              
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libunwindstack.so  
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libunwindstack.so      
warning: section .plt not found in .gnu_debugdata for /system/lib64/libunwindstack.so           
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libunwindstack.so   
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libunwindstack.so    
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libunwindstack.so    
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libunwindstack.so       
warning: section .got not found in .gnu_debugdata for /system/lib64/libunwindstack.so           
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libunwindstack.so       
warning: section .data not found in .gnu_debugdata for /system/lib64/libunwindstack.so          
warning: section .bss not found in .gnu_debugdata for /system/lib64/libunwindstack.so           
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libc++.so    
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libc++.so     
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libc++.so                
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libc++.so           
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libc++.so         
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libc++.so              
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libc++.so                
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libc++.so              
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libc++.so              
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libc++.so              
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libc++.so               
warning: section .gcc_except_table not found in .gnu_debugdata for /system/lib64/libc++.so      
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libc++.so          
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libc++.so              
warning: section .plt not found in .gnu_debugdata for /system/lib64/libc++.so                   
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libc++.so           
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libc++.so            
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libc++.so            
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libc++.so               
warning: section .got not found in .gnu_debugdata for /system/lib64/libc++.so                   
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libc++.so               
warning: section .data not found in .gnu_debugdata for /system/lib64/libc++.so                  
warning: section .bss not found in .gnu_debugdata for /system/lib64/libc++.so                   
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libcutils.so 
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libcutils.so  
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libcutils.so             
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libcutils.so        
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libcutils.so      
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libcutils.so           
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libcutils.so             
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libcutils.so           
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libcutils.so           
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libcutils.so           
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libcutils.so             
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libcutils.so       
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libcutils.so           
warning: section .plt not found in .gnu_debugdata for /system/lib64/libcutils.so                
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libcutils.so        
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libcutils.so         
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libcutils.so         
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libcutils.so            
warning: section .got not found in .gnu_debugdata for /system/lib64/libcutils.so                
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libcutils.so            
warning: section .data not found in .gnu_debugdata for /system/lib64/libcutils.so               
warning: section .bss not found in .gnu_debugdata for /system/lib64/libcutils.so                
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/liblog.so    
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/liblog.so     
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/liblog.so                
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/liblog.so           
warning: section .gnu.version_d not found in .gnu_debugdata for /system/lib64/liblog.so         
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/liblog.so         
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/liblog.so              
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/liblog.so                
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/liblog.so              
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/liblog.so              
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/liblog.so              
warning: section .rodata not found in .gnu_debugdata for /system/lib64/liblog.so                
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/liblog.so          
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/liblog.so              
warning: section .plt not found in .gnu_debugdata for /system/lib64/liblog.so                   
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/liblog.so           
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/liblog.so            
warning: section .init_array not found in .gnu_debugdata for /system/lib64/liblog.so            
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/liblog.so               
warning: section .got not found in .gnu_debugdata for /system/lib64/liblog.so                   
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/liblog.so               
warning: section .data not found in .gnu_debugdata for /system/lib64/liblog.so                  
warning: section .bss not found in .gnu_debugdata for /system/lib64/liblog.so                   
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libvndksupport.so                                           
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libvndksupport.so                                            
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libvndksupport.so        
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libvndksupport.so   
warning: section .gnu.version_d not found in .gnu_debugdata for /system/lib64/libvndksupport.so 
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libvndksupport.so 
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libvndksupport.so      
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libvndksupport.so        
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libvndksupport.so      
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libvndksupport.so      
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libvndksupport.so        
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libvndksupport.so  
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libvndksupport.so      
warning: section .plt not found in .gnu_debugdata for /system/lib64/libvndksupport.so           
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libvndksupport.so   
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libvndksupport.so    
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libvndksupport.so       
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libvndksupport.so       
warning: section .bss not found in .gnu_debugdata for /system/lib64/libvndksupport.so           
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libbase.so   
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libbase.so    
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libbase.so               
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libbase.so          
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libbase.so        
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libbase.so             
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libbase.so               
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libbase.so             
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libbase.so             
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libbase.so             
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libbase.so               
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libbase.so         
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libbase.so             
warning: section .plt not found in .gnu_debugdata for /system/lib64/libbase.so                  
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libbase.so          
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libbase.so           
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libbase.so           
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libbase.so              
warning: section .got not found in .gnu_debugdata for /system/lib64/libbase.so                  
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libbase.so              
warning: section .data not found in .gnu_debugdata for /system/lib64/libbase.so                 
warning: section .bss not found in .gnu_debugdata for /system/lib64/libbase.so                  
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/liblzma.so   
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/liblzma.so    
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/liblzma.so               
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/liblzma.so          
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/liblzma.so        
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/liblzma.so             
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/liblzma.so               
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/liblzma.so             
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/liblzma.so             
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/liblzma.so             
warning: section .rodata not found in .gnu_debugdata for /system/lib64/liblzma.so               
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/liblzma.so         
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/liblzma.so             
warning: section .plt not found in .gnu_debugdata for /system/lib64/liblzma.so                  
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/liblzma.so          
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/liblzma.so           
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/liblzma.so              
warning: section .got not found in .gnu_debugdata for /system/lib64/liblzma.so                  
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/liblzma.so              
warning: section .bss not found in .gnu_debugdata for /system/lib64/liblzma.so                  
warning: section .note.gnu.build-id not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                     
warning: section .dynsym not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                                
warning: section .gnu.version not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                           
warning: section .gnu.version_d not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                         
warning: section .gnu.hash not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                              
warning: section .dynstr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                                
warning: section .rela.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                              
warning: section .eh_frame_hdr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                          
warning: section .eh_frame not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                              
warning: section .plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                                   
warning: section .dynamic not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                               
warning: section .got.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                               
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libnetd_client.so                                            
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libnetd_client.so                                             
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libnetd_client.so        
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libnetd_client.so   
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libnetd_client.so 
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libnetd_client.so      
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libnetd_client.so        
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libnetd_client.so      
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libnetd_client.so      
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libnetd_client.so      
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libnetd_client.so        
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libnetd_client.so  
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libnetd_client.so      
warning: section .plt not found in .gnu_debugdata for /system/lib64/libnetd_client.so           
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libnetd_client.so   
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libnetd_client.so    
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libnetd_client.so       
warning: section .got not found in .gnu_debugdata for /system/lib64/libnetd_client.so           
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libnetd_client.so       
warning: section .data not found in .gnu_debugdata for /system/lib64/libnetd_client.so          
warning: section .bss not found in .gnu_debugdata for /system/lib64/libnetd_client.so           
Log start                                       
main: build = 1 (11dc109)                       
main: built with clang version 17.0.2 for aarch64-unknown-linux-android24                       
main: seed  = 1697346575                        
llama_model_loader: loaded meta data with 19 key-value pairs and 291 tensors from /sdcard/Download/llama-2-7b-chat.Q3_K_S.gguf (version GGUF V2 (latest))                                       
llama_model_loader: - tensor    0:                token_embd.weight q3_K     [  4096, 32000,     1,     1 ]                                     
llama_model_loader: - tensor    1:           blk.0.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor    2:            blk.0.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor    3:            blk.0.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor    4:              blk.0.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor    5:            blk.0.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor    6:              blk.0.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor    7:         blk.0.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor    8:              blk.0.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor    9:              blk.0.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   10:           blk.1.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   11:            blk.1.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   12:            blk.1.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   13:              blk.1.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   14:            blk.1.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   15:              blk.1.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   16:         blk.1.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   17:              blk.1.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   18:              blk.1.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   19:          blk.10.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   20:           blk.10.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   21:           blk.10.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   22:             blk.10.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   23:           blk.10.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   24:             blk.10.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   25:        blk.10.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   26:             blk.10.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   27:             blk.10.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   28:          blk.11.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   29:           blk.11.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   30:           blk.11.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   31:             blk.11.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   32:           blk.11.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   33:             blk.11.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   34:        blk.11.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   35:             blk.11.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   36:             blk.11.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   37:          blk.12.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   38:           blk.12.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   39:           blk.12.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   40:             blk.12.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   41:           blk.12.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   42:             blk.12.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   43:        blk.12.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   44:             blk.12.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   45:             blk.12.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   46:          blk.13.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   47:           blk.13.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   48:           blk.13.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   49:             blk.13.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   50:           blk.13.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   51:             blk.13.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   52:        blk.13.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   53:             blk.13.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   54:             blk.13.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   55:          blk.14.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   56:           blk.14.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   57:           blk.14.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   58:             blk.14.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   59:           blk.14.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   60:             blk.14.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   61:        blk.14.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   62:             blk.14.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   63:             blk.14.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   64:          blk.15.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   65:           blk.15.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   66:           blk.15.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   67:             blk.15.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   68:           blk.15.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   69:             blk.15.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   70:        blk.15.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   71:             blk.15.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   72:             blk.15.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   73:          blk.16.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   74:           blk.16.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   75:           blk.16.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   76:             blk.16.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   77:           blk.16.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   78:             blk.16.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   79:        blk.16.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   80:             blk.16.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   81:             blk.16.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   82:          blk.17.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   83:           blk.17.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   84:           blk.17.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   85:             blk.17.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   86:           blk.17.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   87:             blk.17.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   88:        blk.17.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   89:             blk.17.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   90:             blk.17.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   91:          blk.18.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   92:           blk.18.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   93:           blk.18.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   94:             blk.18.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   95:           blk.18.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   96:             blk.18.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   97:        blk.18.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   98:             blk.18.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   99:             blk.18.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  100:          blk.19.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  101:           blk.19.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  102:           blk.19.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  103:             blk.19.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  104:           blk.19.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  105:             blk.19.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  106:        blk.19.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  107:             blk.19.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  108:             blk.19.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  109:           blk.2.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  110:            blk.2.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  111:            blk.2.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  112:              blk.2.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  113:            blk.2.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  114:              blk.2.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  115:         blk.2.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  116:              blk.2.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  117:              blk.2.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  118:          blk.20.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  119:           blk.20.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  120:           blk.20.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  121:             blk.20.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  122:           blk.20.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  123:             blk.20.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  124:        blk.20.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  125:             blk.20.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  126:             blk.20.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  127:          blk.21.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  128:           blk.21.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  129:           blk.21.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  130:             blk.21.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  131:           blk.21.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  132:             blk.21.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  133:        blk.21.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  134:             blk.21.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  135:             blk.21.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  136:          blk.22.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  137:           blk.22.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  138:           blk.22.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  139:             blk.22.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  140:           blk.22.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  141:             blk.22.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  142:        blk.22.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  143:             blk.22.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  144:             blk.22.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  145:          blk.23.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  146:           blk.23.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  147:           blk.23.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  148:             blk.23.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  149:           blk.23.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  150:             blk.23.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  151:        blk.23.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  152:             blk.23.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  153:             blk.23.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  154:           blk.3.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  155:            blk.3.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  156:            blk.3.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  157:              blk.3.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  158:            blk.3.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  159:              blk.3.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  160:         blk.3.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  161:              blk.3.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  162:              blk.3.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  163:           blk.4.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  164:            blk.4.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  165:            blk.4.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  166:              blk.4.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  167:            blk.4.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  168:              blk.4.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  169:         blk.4.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  170:              blk.4.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  171:              blk.4.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  172:           blk.5.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  173:            blk.5.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  174:            blk.5.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  175:              blk.5.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  176:            blk.5.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  177:              blk.5.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  178:         blk.5.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  179:              blk.5.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  180:              blk.5.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  181:           blk.6.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  182:            blk.6.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  183:            blk.6.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  184:              blk.6.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  185:            blk.6.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  186:              blk.6.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  187:         blk.6.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  188:              blk.6.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  189:              blk.6.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  190:           blk.7.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  191:            blk.7.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  192:            blk.7.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  193:              blk.7.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  194:            blk.7.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  195:              blk.7.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  196:         blk.7.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  197:              blk.7.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  198:              blk.7.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  199:           blk.8.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  200:            blk.8.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  201:            blk.8.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  202:              blk.8.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  203:            blk.8.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  204:              blk.8.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  205:         blk.8.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  206:              blk.8.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  207:              blk.8.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  208:           blk.9.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  209:            blk.9.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  210:            blk.9.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  211:              blk.9.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  212:            blk.9.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  213:              blk.9.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  214:         blk.9.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  215:              blk.9.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  216:              blk.9.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  217:                    output.weight q6_K     [  4096, 32000,     1,     1 ]                                     
llama_model_loader: - tensor  218:          blk.24.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  219:           blk.24.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  220:           blk.24.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  221:             blk.24.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  222:           blk.24.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  223:             blk.24.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  224:        blk.24.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  225:             blk.24.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  226:             blk.24.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  227:          blk.25.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  228:           blk.25.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  229:           blk.25.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  230:             blk.25.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  231:           blk.25.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  232:             blk.25.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  233:        blk.25.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  234:             blk.25.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  235:             blk.25.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  236:          blk.26.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  237:           blk.26.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  238:           blk.26.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  239:             blk.26.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  240:           blk.26.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  241:             blk.26.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  242:        blk.26.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  243:             blk.26.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  244:             blk.26.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  245:          blk.27.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  246:           blk.27.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  247:           blk.27.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  248:             blk.27.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  249:           blk.27.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  250:             blk.27.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  251:        blk.27.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  252:             blk.27.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  253:             blk.27.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  254:          blk.28.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  255:           blk.28.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  256:           blk.28.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  257:             blk.28.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  258:           blk.28.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  259:             blk.28.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  260:        blk.28.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  261:             blk.28.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  262:             blk.28.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  263:          blk.29.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  264:           blk.29.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  265:           blk.29.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  266:             blk.29.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  267:           blk.29.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  268:             blk.29.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  269:        blk.29.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  270:             blk.29.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  271:             blk.29.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  272:          blk.30.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  273:           blk.30.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  274:           blk.30.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  275:             blk.30.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  276:           blk.30.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  277:             blk.30.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  278:        blk.30.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  279:             blk.30.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  280:             blk.30.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  281:          blk.31.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  282:           blk.31.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  283:           blk.31.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  284:             blk.31.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  285:           blk.31.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  286:             blk.31.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  287:        blk.31.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  288:             blk.31.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  289:             blk.31.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  290:               output_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - kv   0:                       general.architecture str                    
llama_model_loader: - kv   1:                               general.name str                    
llama_model_loader: - kv   2:                       llama.context_length u32                    
llama_model_loader: - kv   3:                     llama.embedding_length u32                    
llama_model_loader: - kv   4:                          llama.block_count u32                    
llama_model_loader: - kv   5:                  llama.feed_forward_length u32                    
llama_model_loader: - kv   6:                 llama.rope.dimension_count u32                    
llama_model_loader: - kv   7:                 llama.attention.head_count u32                    
llama_model_loader: - kv   8:              llama.attention.head_count_kv u32                    
llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32                    
llama_model_loader: - kv  10:                          general.file_type u32                    
llama_model_loader: - kv  11:                       tokenizer.ggml.model str                    
llama_model_loader: - kv  12:                      tokenizer.ggml.tokens arr                    
llama_model_loader: - kv  13:                      tokenizer.ggml.scores arr                    
llama_model_loader: - kv  14:                  tokenizer.ggml.token_type arr                    
llama_model_loader: - kv  15:                tokenizer.ggml.bos_token_id u32                    
llama_model_loader: - kv  16:                tokenizer.ggml.eos_token_id u32                    
llama_model_loader: - kv  17:            tokenizer.ggml.unknown_token_id u32                    
llama_model_loader: - kv  18:               general.quantization_version u32                    
llama_model_loader: - type  f32:   65 tensors   
llama_model_loader: - type q3_K:  225 tensors   
llama_model_loader: - type q6_K:    1 tensors   
llm_load_print_meta: format           = GGUF V2 (latest)                                        
llm_load_print_meta: arch             = llama   
llm_load_print_meta: vocab type       = SPM     
llm_load_print_meta: n_vocab          = 32000   
llm_load_print_meta: n_merges         = 0       
llm_load_print_meta: n_ctx_train      = 4096    
llm_load_print_meta: n_embd           = 4096    
llm_load_print_meta: n_head           = 32      
llm_load_print_meta: n_head_kv        = 32      
llm_load_print_meta: n_layer          = 32      
llm_load_print_meta: n_rot            = 128     
llm_load_print_meta: n_gqa            = 1       
llm_load_print_meta: f_norm_eps       = 0.0e+00 
llm_load_print_meta: f_norm_rms_eps   = 1.0e-06 
llm_load_print_meta: f_clamp_kqv      = 0.0e+00 
llm_load_print_meta: f_max_alibi_bias = 0.0e+00 
llm_load_print_meta: n_ff             = 11008   
llm_load_print_meta: freq_base_train  = 10000.0 
llm_load_print_meta: freq_scale_train = 1       
llm_load_print_meta: model type       = 7B      
llm_load_print_meta: model ftype      = mostly Q3_K - Small                                     
llm_load_print_meta: model params     = 6.74 B  
llm_load_print_meta: model size       = 2.75 GiB (3.50 BPW)                                     
llm_load_print_meta: general.name   = LLaMA v2  
llm_load_print_meta: BOS token = 1 '<s>'        
llm_load_print_meta: EOS token = 2 '</s>'       
llm_load_print_meta: UNK token = 0 '<unk>'      
llm_load_print_meta: LF token  = 13 '<0x0A>'    
llm_load_tensors: ggml ctx size =    0.10 MB    
llm_load_tensors: mem required  = 2811.11 MB    
.................................................................................................                                               
llama_new_context_with_model: n_ctx      = 2048 
llama_new_context_with_model: freq_base  = 10000.0                                              
llama_new_context_with_model: freq_scale = 1    
llama_new_context_with_model: kv self size  = 1024.00 MB                                        
llama_new_context_with_model: compute buffer total size = 9.17 MB                               
[New Thread 0x137d (LWP 4989)]                  
[New Thread 0x137e (LWP 4990)]                                                                  
[Thread 0x137e (LWP 4990) exited]               
[Thread 0x137d (LWP 4989) exited]               
system_info: n_threads = 3 / 8 | AVX = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 |                  
main: interactive mode on.                      
Reverse prompt: '### Instruction:                                                               '                                               sampling: repeat_last_n = 64, repeat_penalty = 1.100000, presence_penalty = 0.000000, frequency_penalty = 0.000000, top_k = 40, tfs_z = 1.000000, top_p = 0.950000, typical_p = 1.000000, temp = 0.800000, mirostat = 0, mirostat_lr = 0.100000, mirostat_ent = 5.000000                        
generate: n_ctx = 2048, n_batch = 10, n_predict = -1, n_keep = 1

== Running in interactive mode. ==               
- Press Ctrl+C to interject at any time.        
- Press Return to return control to LLaMa.      
- To return control without starting a new line, end your input with '/'.                       
- If you want to submit another line, end your input with '\'.                                                                                 
[New Thread 0x1380 (LWP 4992)]                  
[New Thread 0x1381 (LWP 4993)]                                                                  
> [Thread 0x1381 (LWP 4993) exited]             
[Thread 0x1380 (LWP 4992) exited]                                                                
Hinweis[New Thread 0x138b (LWP 5003)]          
[New Thread 0x138c (LWP 5004)]                 
[Thread 0x138c (LWP 5004) exited]             
[Thread 0x138b (LWP 5003) exited]               
,[New Thread 0x138e (LWP 5006)]                 
[New Thread 0x138f (LWP 5007)]                  
[Thread 0x138f (LWP 5007) exited]               
[Thread 0x138e (LWP 5006) exited]                
I[New Thread 0x1398 (LWP 5016)]                
[New Thread 0x1399 (LWP 5017)]                  
[Thread 0x1399 (LWP 5017) exited]               
[Thread 0x1398 (LWP 5016) exited]                
am[New Thread 0x139b (LWP 5019)]               
[New Thread 0x139c (LWP 5020)]                  
[Thread 0x139c (LWP 5020) exited]               
[Thread 0x139b (LWP 5019) exited]                
not[New Thread 0x139e (LWP 5022)]              
[New Thread 0x139f (LWP 5023)]                  
[Thread 0x139f (LWP 5023) exited]               
[Thread 0x139e (LWP 5022) exited]                
able[New Thread 0x13a1 (LWP 5025)]             
[New Thread 0x13a2 (LWP 5026)]                  
[Thread 0x13a2 (LWP 5026) exited]               
[Thread 0x13a1 (LWP 5025) exited]                
to[New Thread 0x13a4 (LWP 5028)]               
[New Thread 0x13a7 (LWP 5031)]                  
[Thread 0x13a7 (LWP 5031) exited]               
[Thread 0x13a4 (LWP 5028) exited]                
answer[New Thread 0x13a9 (LWP 5033)]           
[New Thread 0x13aa (LWP 5034)]                  
[Thread 0x13aa (LWP 5034) exited]               
[Thread 0x13a9 (LWP 5033) exited]                
all[New Thread 0x13ad (LWP 5037)]              
[New Thread 0x13ae (LWP 5038)]                  
[Thread 0x13ae (LWP 5038) exited]               
[Thread 0x13ad (LWP 5037) exited]                
of[New Thread 0x13b4 (LWP 5044)]               
[New Thread 0x13d1 (LWP 5073)]                  
[Thread 0x13d1 (LWP 5073) exited]               
[Thread 0x13b4 (LWP 5044) exited]                
the[New Thread 0x13de (LWP 5086)]              
[New Thread 0x13df (LWP 5087)]                  
[Thread 0x13df (LWP 5087) exited]               
[Thread 0x13de (LWP 5086) exited]                
questions[New Thread 0x13e1 (LWP 5089)]        
[New Thread 0x13e2 (LWP 5090)]                  
[Thread 0x13e2 (LWP 5090) exited]               
[Thread 0x13e1 (LWP 5089) exited]                
in[New Thread 0x13e6 (LWP 5094)]               
[New Thread 0x13e7 (LWP 5095)]                  
[Thread 0x13e7 (LWP 5095) exited]               
[Thread 0x13e6 (LWP 5094) exited]                
the[New Thread 0x13e9 (LWP 5097)]              
[New Thread 0x13ea (LWP 5098)]                  
[Thread 0x13ea (LWP 5098) exited]               
[Thread 0x13e9 (LWP 5097) exited]                
qu[New Thread 0x13ec (LWP 5100)]               
[New Thread 0x13ed (LWP 5101)]                  
[Thread 0x13ed (LWP 5101) exited]               
[Thread 0x13ec (LWP 5100) exited]               
iz[New Thread 0x13ef (LWP 5103)]                
[New Thread 0x13f0 (LWP 5104)]                  
[Thread 0x13f0 (LWP 5104) exited]               
[Thread 0x13ef (LWP 5103) exited]                
as[New Thread 0x13f2 (LWP 5106)]               
[New Thread 0x13f3 (LWP 5107)]                  
[Thread 0x13f3 (LWP 5107) exited]               
[Thread 0x13f2 (LWP 5106) exited]                
some[New Thread 0x13f5 (LWP 5109)]             
[New Thread 0x13f6 (LWP 5110)]                  
[Thread 0x13f6 (LWP 5110) exited]               
[Thread 0x13f5 (LWP 5109) exited]                
of[New Thread 0x13f8 (LWP 5112)]               
[New Thread 0x13f9 (LWP 5113)]                  
[Thread 0x13f9 (LWP 5113) exited]               
[Thread 0x13f8 (LWP 5112) exited]                
the[New Thread 0x13fb (LWP 5115)]              
[New Thread 0x13fc (LWP 5116)]                  
[Thread 0x13fc (LWP 5116) exited]               
[Thread 0x13fb (LWP 5115) exited]                
information[New Thread 0x13fe (LWP 5118)]      
[New Thread 0x13ff (LWP 5119)]                  
[Thread 0x13ff (LWP 5119) exited]               
[Thread 0x13fe (LWP 5118) exited]                
is[New Thread 0x1401 (LWP 5121)]               
[New Thread 0x1402 (LWP 5122)]                  
[Thread 0x1402 (LWP 5122) exited]               
[Thread 0x1401 (LWP 5121) exited]                
not[New Thread 0x1404 (LWP 5124)]              
[New Thread 0x1405 (LWP 5125)]                  
[Thread 0x1405 (LWP 5125) exited]               
[Thread 0x1404 (LWP 5124) exited]                
available[New Thread 0x1407 (LWP 5127)]        
[New Thread 0x1408 (LWP 5128)]                  
[Thread 0x1408 (LWP 5128) exited]               
[Thread 0x1407 (LWP 5127) exited]                
to[New Thread 0x140a (LWP 5130)]               
[New Thread 0x140b (LWP 5131)]                  
[Thread 0x140b (LWP 5131) exited]                
me[Thread 0x140a (LWP 5130) exited]            
[New Thread 0x140d (LWP 5133)]                  
[New Thread 0x140e (LWP 5134)]                  
[Thread 0x140e (LWP 5134) exited]               
[Thread 0x140d (LWP 5133) exited]               
.[New Thread 0x1410 (LWP 5136)]                 
[New Thread 0x1411 (LWP 5137)]                  
[Thread 0x1411 (LWP 5137) exited]                
nobody[Thread 0x1410 (LWP 5136) exited]        
[New Thread 0x1413 (LWP 5139)]                  
[New Thread 0x1414 (LWP 5140)]                  
[Thread 0x1414 (LWP 5140) exited]               
[Thread 0x1413 (LWP 5139) exited]                
has[New Thread 0x1416 (LWP 5142)]              
[New Thread 0x1417 (LWP 5143)]                  
[Thread 0x1417 (LWP 5143) exited]               
[Thread 0x1416 (LWP 5142) exited]                
been[New Thread 0x1419 (LWP 5145)]             
[New Thread 0x141a (LWP 5146)]                  
[Thread 0x141a (LWP 5146) exited]               
[Thread 0x1419 (LWP 5145) exited]                
able[New Thread 0x141c (LWP 5148)]             
[New Thread 0x141d (LWP 5149)]                  
[Thread 0x141d (LWP 5149) exited]               
[Thread 0x141c (LWP 5148) exited]                
to[New Thread 0x141f (LWP 5151)]               
[New Thread 0x1420 (LWP 5152)]                  
[Thread 0x1420 (LWP 5152) exited]               
[Thread 0x141f (LWP 5151) exited]                
create[New Thread 0x1422 (LWP 5154)]           
[New Thread 0x1423 (LWP 5155)]                  
[Thread 0x1423 (LWP 5155) exited]               
[Thread 0x1422 (LWP 5154) exited]                
a[New Thread 0x1425 (LWP 5157)]                
[New Thread 0x1426 (LWP 5158)]                  
[Thread 0x1426 (LWP 5158) exited]               
[Thread 0x1425 (LWP 5157) exited]                
vi[New Thread 0x142a (LWP 5162)]               
[New Thread 0x142b (LWP 5163)]                  
[Thread 0x142b (LWP 5163) exited]               
[Thread 0x142a (LWP 5162) exited]               
able[New Thread 0x1431 (LWP 5169)]              
[New Thread 0x1432 (LWP 5170)]                  
[Thread 0x1432 (LWP 5170) exited]               
[Thread 0x1431 (LWP 5169) exited]                
A[New Thread 0x1434 (LWP 5172)]                
[New Thread 0x1435 (LWP 5173)]                  
[Thread 0x1435 (LWP 5173) exited]               
[Thread 0x1434 (LWP 5172) exited]               
I[New Thread 0x1437 (LWP 5175)]                 
[New Thread 0x1439 (LWP 5177)]                  
[Thread 0x1439 (LWP 5177) exited]               
[Thread 0x1437 (LWP 5175) exited]                
language[New Thread 0x143c (LWP 5180)]         
[New Thread 0x143d (LWP 5181)]                  
[Thread 0x143d (LWP 5181) exited]               
[Thread 0x143c (LWP 5180) exited]                
model[New Thread 0x143f (LWP 5183)]            
[New Thread 0x1440 (LWP 5184)]                  
[Thread 0x1440 (LWP 5184) exited]               
[Thread 0x143f (LWP 5183) exited]                
that[New Thread 0x1445 (LWP 5189)]             
[New Thread 0x1447 (LWP 5191)]                  
[Thread 0x1447 (LWP 5191) exited]                
can[Thread 0x1445 (LWP 5189) exited]           
[New Thread 0x144a (LWP 5194)]                  
[New Thread 0x144c (LWP 5196)]                  
[Thread 0x144c (LWP 5196) exited]               
[Thread 0x144a (LWP 5194) exited]                
understand[New Thread 0x1450 (LWP 5200)]       
[New Thread 0x1451 (LWP 5201)]                  
[Thread 0x1451 (LWP 5201) exited]               
[Thread 0x1450 (LWP 5200) exited]                
and[New Thread 0x1453 (LWP 5203)]              
[New Thread 0x1454 (LWP 5204)]                  
[Thread 0x1454 (LWP 5204) exited]               
[Thread 0x1453 (LWP 5203) exited]                
respond[New Thread 0x1457 (LWP 5207)]          
[New Thread 0x1459 (LWP 5209)]                  
[Thread 0x1459 (LWP 5209) exited]               
[Thread 0x1457 (LWP 5207) exited]                
to[New Thread 0x145b (LWP 5211)]               
[New Thread 0x145c (LWP 5212)]                  
[Thread 0x145c (LWP 5212) exited]               
[Thread 0x145b (LWP 5211) exited]                
complex[New Thread 0x145f (LWP 5215)]          
[New Thread 0x1460 (LWP 5216)]                  
[Thread 0x1460 (LWP 5216) exited]               
[Thread 0x145f (LWP 5215) exited]                
queries[New Thread 0x1464 (LWP 5220)]          
[New Thread 0x1465 (LWP 5221)]                  
[Thread 0x1465 (LWP 5221) exited]               
[Thread 0x1464 (LWP 5220) exited]                
like[New Thread 0x1467 (LWP 5223)]             
[New Thread 0x1468 (LWP 5224)]                  
[Thread 0x1468 (LWP 5224) exited]               
[Thread 0x1467 (LWP 5223) exited]                
a[New Thread 0x146a (LWP 5226)]                
[New Thread 0x146b (LWP 5227)]                  
[Thread 0x146b (LWP 5227) exited]               
[Thread 0x146a (LWP 5226) exited]                
human[New Thread 0x146e (LWP 5230)]            
[New Thread 0x146f (LWP 5231)]                  
[Thread 0x146f (LWP 5231) exited]               
[Thread 0x146e (LWP 5230) exited]                
would[New Thread 0x1472 (LWP 5234)]            
[New Thread 0x1474 (LWP 5236)]                  
[Thread 0x1474 (LWP 5236) exited]               
[Thread 0x1472 (LWP 5234) exited]               
.[New Thread 0x1476 (LWP 5238)]                 
[New Thread 0x1477 (LWP 5239)]                  
[Thread 0x1477 (LWP 5239) exited]               
[Thread 0x1476 (LWP 5238) exited]                                                               
[New Thread 0x1479 (LWP 5241)]                  
[New Thread 0x147a (LWP 5242)]                  
[Thread 0x147a (LWP 5242) exited]               
[Thread 0x1479 (LWP 5241) exited]               
However[New Thread 0x147c (LWP 5244)]           
[New Thread 0x147d (LWP 5245)]                  
[Thread 0x147d (LWP 5245) exited]               
[Thread 0x147c (LWP 5244) exited]               
,[New Thread 0x147f (LWP 5247)]                 
[New Thread 0x1480 (LWP 5248)]                  
[Thread 0x1480 (LWP 5248) exited]               
[Thread 0x147f (LWP 5247) exited]                
I[New Thread 0x1485 (LWP 5253)]                
[New Thread 0x1486 (LWP 5254)]                  
[Thread 0x1486 (LWP 5254) exited]               
[Thread 0x1485 (LWP 5253) exited]                
can[New Thread 0x1498 (LWP 5272)]              
[New Thread 0x1499 (LWP 5273)]                  
[Thread 0x1499 (LWP 5273) exited]               
[Thread 0x1498 (LWP 5272) exited]                
try[New Thread 0x14ad (LWP 5293)]              
[New Thread 0x14ae (LWP 5294)]                  
[Thread 0x14ae (LWP 5294) exited]               
[Thread 0x14ad (LWP 5293) exited]                
to[New Thread 0x14f7 (LWP 5367)]               
[New Thread 0x14f8 (LWP 5368)]                  
[Thread 0x14f8 (LWP 5368) exited]               
[Thread 0x14f7 (LWP 5367) exited]                
help[New Thread 0x1520 (LWP 5408)]             
[New Thread 0x1521 (LWP 5409)]                  
[Thread 0x1521 (LWP 5409) exited]               
[Thread 0x1520 (LWP 5408) exited]                
you[New Thread 0x1525 (LWP 5413)]              
[New Thread 0x1526 (LWP 5414)]                  
[Thread 0x1526 (LWP 5414) exited]               
[Thread 0x1525 (LWP 5413) exited]                
with[New Thread 0x1545 (LWP 5445)]             
[New Thread 0x1546 (LWP 5446)]                  
[Thread 0x1546 (LWP 5446) exited]               
[Thread 0x1545 (LWP 5445) exited]                
any[New Thread 0x15a1 (LWP 5537)]              
[New Thread 0x15a2 (LWP 5538)]                  
[Thread 0x15a2 (LWP 5538) exited]               
[Thread 0x15a1 (LWP 5537) exited]                
specific[New Thread 0x172f (LWP 5935)]         
[New Thread 0x1730 (LWP 5936)]                  
[Thread 0x1730 (LWP 5936) exited]               
[Thread 0x172f (LWP 5935) exited]                
questions[New Thread 0x1781 (LWP 6017)]        
[New Thread 0x1782 (LWP 6018)]                  
[Thread 0x1782 (LWP 6018) exited]               
[Thread 0x1781 (LWP 6017) exited]                
or[New Thread 0x17a5 (LWP 6053)]               
[New Thread 0x17a7 (LWP 6055)]                  
[Thread 0x17a7 (LWP 6055) exited]               
[Thread 0x17a5 (LWP 6053) exited]                
topics[New Thread 0x17d4 (LWP 6100)]           
[New Thread 0x17d5 (LWP 6101)]                  
[Thread 0x17d5 (LWP 6101) exited]               
[Thread 0x17d4 (LWP 6100) exited]                
you[New Thread 0x17ea (LWP 6122)]              
[New Thread 0x17eb (LWP 6123)]                  
[Thread 0x17eb (LWP 6123) exited]               
[Thread 0x17ea (LWP 6122) exited]                
have[New Thread 0x17f8 (LWP 6136)]             
[New Thread 0x17f9 (LWP 6137)]                  
[Thread 0x17f9 (LWP 6137) exited]               
[Thread 0x17f8 (LWP 6136) exited]                
in[New Thread 0x180d (LWP 6157)]               
[New Thread 0x180e (LWP 6158)]                  
[Thread 0x180e (LWP 6158) exited]               
[Thread 0x180d (LWP 6157) exited]                
mind[New Thread 0x1816 (LWP 6166)]             
[New Thread 0x1817 (LWP 6167)]                  
[Thread 0x1817 (LWP 6167) exited]               
[Thread 0x1816 (LWP 6166) exited]               
.[New Thread 0x1820 (LWP 6176)]                 
[New Thread 0x1821 (LWP 6177)]                  
[Thread 0x1821 (LWP 6177) exited]               
[Thread 0x1820 (LWP 6176) exited]                
Just[New Thread 0x182f (LWP 6191)]             
[New Thread 0x1830 (LWP 6192)]                  
[Thread 0x1830 (LWP 6192) exited]               
[Thread 0x182f (LWP 6191) exited]                
let[New Thread 0x185e (LWP 6238)]              
[New Thread 0x1861 (LWP 6241)]                  
[Thread 0x1861 (LWP 6241) exited]               
[Thread 0x185e (LWP 6238) exited]                
me[New Thread 0x1877 (LWP 6263)]               
[New Thread 0x1879 (LWP 6265)]                  
[Thread 0x1879 (LWP 6265) exited]               
[Thread 0x1877 (LWP 6263) exited]                
know[New Thread 0x1883 (LWP 6275)]             
[New Thread 0x1884 (LWP 6276)]                  
[Thread 0x1884 (LWP 6276) exited]               
[Thread 0x1883 (LWP 6275) exited]                
what[New Thread 0x188a (LWP 6282)]             
[New Thread 0x188b (LWP 6283)]                  
[Thread 0x188b (LWP 6283) exited]               
[Thread 0x188a (LWP 6282) exited]               
'[New Thread 0x188d (LWP 6285)]                 
[New Thread 0x188e (LWP 6286)]                  
[Thread 0x188e (LWP 6286) exited]               
[Thread 0x188d (LWP 6285) exited]               
s[New Thread 0x1893 (LWP 6291)]                 
[New Thread 0x1894 (LWP 6292)]                  
[Thread 0x1894 (LWP 6292) exited]               
[Thread 0x1893 (LWP 6291) exited]                
on[New Thread 0x18a8 (LWP 6312)]               
[New Thread 0x18a9 (LWP 6313)]                  
[Thread 0x18a9 (LWP 6313) exited]               
[Thread 0x18a8 (LWP 6312) exited]                
your[New Thread 0x18b4 (LWP 6324)]             
[New Thread 0x18b5 (LWP 6325)]                  
[Thread 0x18b5 (LWP 6325) exited]               
[Thread 0x18b4 (LWP 6324) exited]                
mind[New Thread 0x18c3 (LWP 6339)]             
[New Thread 0x18c4 (LWP 6340)]                  
[Thread 0x18c4 (LWP 6340) exited]               
[Thread 0x18c3 (LWP 6339) exited]               
,[New Thread 0x18c7 (LWP 6343)]                 
[New Thread 0x18c8 (LWP 6344)]                  
[Thread 0x18c8 (LWP 6344) exited]               
[Thread 0x18c7 (LWP 6343) exited]                
and[New Thread 0x18cb (LWP 6347)]              
[New Thread 0x18cc (LWP 6348)]                  
[Thread 0x18cc (LWP 6348) exited]               
[Thread 0x18cb (LWP 6347) exited]                
I[New Thread 0x18d1 (LWP 6353)]                
[New Thread 0x18d2 (LWP 6354)]                  
[Thread 0x18d2 (LWP 6354) exited]               
[Thread 0x18d1 (LWP 6353) exited]               
'[New Thread 0x18d5 (LWP 6357)]                 
[New Thread 0x18d6 (LWP 6358)]                  
[Thread 0x18d6 (LWP 6358) exited]               
[Thread 0x18d5 (LWP 6357) exited]               
ll[New Thread 0x18d9 (LWP 6361)]                
[New Thread 0x18da (LWP 6362)]                  
[Thread 0x18da (LWP 6362) exited]               
[Thread 0x18d9 (LWP 6361) exited]                
do[New Thread 0x18dc (LWP 6364)]               
[New Thread 0x18dd (LWP 6365)]                  
[Thread 0x18dd (LWP 6365) exited]               
[Thread 0x18dc (LWP 6364) exited]                
my[New Thread 0x18df (LWP 6367)]               
[New Thread 0x18e0 (LWP 6368)]                  
[Thread 0x18e0 (LWP 6368) exited]               
[Thread 0x18df (LWP 6367) exited]                
best[New Thread 0x18e4 (LWP 6372)]             
[New Thread 0x18e5 (LWP 6373)]                  
[Thread 0x18e5 (LWP 6373) exited]               
[Thread 0x18e4 (LWP 6372) exited]                
to[New Thread 0x18e9 (LWP 6377)]               
[New Thread 0x18ea (LWP 6378)]                  
[Thread 0x18ea (LWP 6378) exited]               
[Thread 0x18e9 (LWP 6377) exited]                
provide[New Thread 0x18ec (LWP 6380)]          
[New Thread 0x18ed (LWP 6381)]                  
[Thread 0x18ed (LWP 6381) exited]               
[Thread 0x18ec (LWP 6380) exited]                
helpful[New Thread 0x18f2 (LWP 6386)]          
[New Thread 0x18f3 (LWP 6387)]                  
[Thread 0x18f3 (LWP 6387) exited]               
[Thread 0x18f2 (LWP 6386) exited]                
and[New Thread 0x18f8 (LWP 6392)]              
[New Thread 0x18f9 (LWP 6393)]                  
[Thread 0x18f9 (LWP 6393) exited]               
[Thread 0x18f8 (LWP 6392) exited]                
accurate[New Thread 0x18fd (LWP 6397)]         
[New Thread 0x18fe (LWP 6398)]                  
[Thread 0x18fe (LWP 6398) exited]               
[Thread 0x18fd (LWP 6397) exited]                
information[New Thread 0x1901 (LWP 6401)]      
[New Thread 0x1902 (LWP 6402)]                  
[Thread 0x1902 (LWP 6402) exited]               
[Thread 0x1901 (LWP 6401) exited]               
.[New Thread 0x1908 (LWP 6408)]                 
[New Thread 0x1909 (LWP 6409)]                  
[Thread 0x1909 (LWP 6409) exited]               
[Thread 0x1908 (LWP 6408) exited]                                                                                                               
> hi                                            
[New Thread 0x1929 (LWP 6441)]                  
[New Thread 0x192a (LWP 6442)]                  
[Thread 0x192a (LWP 6442) exited]               
[Thread 0x1929 (LWP 6441) exited]               
[New Thread 0x192d (LWP 6445)]                  
[New Thread 0x192e (LWP 6446)]                  
[Thread 0x192e (LWP 6446) exited]               
[New Thread 0x1931 (LWP 6449)]                  
[Thread 0x192d (LWP 6445) exited]               
[New Thread 0x1932 (LWP 6450)]                  
[Thread 0x1932 (LWP 6450) exited]               
[Thread 0x1931 (LWP 6449) exited]               
Hello[New Thread 0x1936 (LWP 6454)]             
[New Thread 0x1937 (LWP 6455)]                  
[Thread 0x1937 (LWP 6455) exited]               
[Thread 0x1936 (LWP 6454) exited]               
![New Thread 0x1939 (LWP 6457)]                 
[New Thread 0x193a (LWP 6458)]                  
[Thread 0x193a (LWP 6458) exited]               
[Thread 0x1939 (LWP 6457) exited]                
It[New Thread 0x193c (LWP 6460)]               
[New Thread 0x193d (LWP 6461)]                  
[Thread 0x193d (LWP 6461) exited]               
[Thread 0x193c (LWP 6460) exited]               
'[New Thread 0x193f (LWP 6463)]                 
[New Thread 0x1940 (LWP 6464)]                  
[Thread 0x1940 (LWP 6464) exited]               
[Thread 0x193f (LWP 6463) exited]               
s[New Thread 0x1942 (LWP 6466)]                 
[New Thread 0x1943 (LWP 6467)]                  
[Thread 0x1943 (LWP 6467) exited]               
[Thread 0x1942 (LWP 6466) exited]                
nice[New Thread 0x1945 (LWP 6469)]             
[New Thread 0x1946 (LWP 6470)]                  
[Thread 0x1946 (LWP 6470) exited]               
[Thread 0x1945 (LWP 6469) exited]                
to[New Thread 0x1948 (LWP 6472)]               
[New Thread 0x1949 (LWP 6473)]                  
[Thread 0x1949 (LWP 6473) exited]               
[Thread 0x1948 (LWP 6472) exited]                
meet[New Thread 0x194b (LWP 6475)]             
[New Thread 0x194c (LWP 6476)]                  
[Thread 0x194c (LWP 6476) exited]               
[Thread 0x194b (LWP 6475) exited]                
you[New Thread 0x194f (LWP 6479)]              
[New Thread 0x1950 (LWP 6480)]                  
[Thread 0x1950 (LWP 6480) exited]               
[Thread 0x194f (LWP 6479) exited]               
.[New Thread 0x1957 (LWP 6487)]                 
[New Thread 0x1958 (LWP 6488)]                                                                  
Thread 1 "main" received signal SIGINT, Interrupt.                                              
0x00000055555fb178 in ggml_graph_compute_thread (data=data@entry=0x7ffffecd60) at /data/data/com.termux/files/home/llama.cpp/ggml.c:18358       18358                   
node_n = atomic_load(&state->shared->node_n);                           
(gdb) bt                                        
#0  0x00000055555fb178 in ggml_graph_compute_thread (data=data@entry=0x7ffffecd60) at /data/data/com.termux/files/home/llama.cpp/ggml.c:18358   
#1  0x00000055555faeec in ggml_graph_compute (cgraph=0xb400007c5bbb7fc0, cplan=<optimized out>) at /data/data/com.termux/files/home/llama.cpp/ggml.c:18903                                      
#2  0x00000055555c83d0 in ggml_graph_compute_helper (buf=..., graph=graph@entry=0xb400007c5bbb7fc0, n_threads=<optimized out>, n_threads@entry=3) at /data/data/com.termux/files/home/llama.cpp/llama.cpp:567                                   
#3  0x00000055555c8e28 in llama_decode_internal (lctx=..., batch=...) at /data/data/com.termux/files/home/llama.cpp/llama.cpp:5868              
#4  0x00000055555c947c in llama_decode (ctx=0x2, batch=...) at /data/data/com.termux/files/home/llama.cpp/llama.cpp:9351                        
#5  0x000000555558cd38 in main (argc=<optimized out>, argv=<optimized out>) at /data/data/com.termux/files/home/llama.cpp/examples/main/main.cpp:602                                            
(gdb) quit                                      
A debugging session is active.                                                                          
Inferior 1 [process 4985] will be killed.                                                                                               
Quit anyway? (y or n) y                         
~/llama.cpp $

I entered ctrl+c to exit. I kept getting a lot of the [New Thread] but at one point you'll see a carrot (>) appeared and allowed me to type. Then, there were responses interspersed within the [New Thread] lines.

theoctopusride commented 12 months ago

ok re-cloned and compiled using $ cmake .. && cmake --build . --config Debug

ran with gdb using ./build/bin/main -m /sdcard/Download/llama-2-7b-chat.Q3_K_S.gguf --color -c 2048 --keep 1 -t 3 -b 10 -i -ins

Starting program: /data/data/com.termux/files/home/llama.cpp/build/bin/main -m /sdcard/Download/llama-2-7b-chat.Q3_K_S.gguf --color -c 2048 --keep 1 -t 3 -b 10 -i -ins
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/data/data/com.termux/files/usr/lib/libthread_db.so".
...
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so    
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so  
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so  
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so  
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so    
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so                                              
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so  
warning: section .plt not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so       
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so                                               
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so   
warning: section .got not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so       
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so   
warning: section .data not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so      
warning: section .bss not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so       
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libc_secshared.so                                            
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libc_secshared.so                                             
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libc_secshared.so        
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libc_secshared.so   
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libc_secshared.so 
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libc_secshared.so      
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libc_secshared.so        
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libc_secshared.so      
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libc_secshared.so      
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libc_secshared.so      
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libc_secshared.so        
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libc_secshared.so  
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libc_secshared.so      
warning: section .plt not found in .gnu_debugdata for /system/lib64/libc_secshared.so           
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libc_secshared.so   
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libc_secshared.so    
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libc_secshared.so       
warning: section .got not found in .gnu_debugdata for /system/lib64/libc_secshared.so           
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libc_secshared.so       
warning: section .data not found in .gnu_debugdata for /system/lib64/libc_secshared.so          
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libutils.so  
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libutils.so   
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libutils.so              
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libutils.so         
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libutils.so       
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libutils.so            
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libutils.so              
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libutils.so            
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libutils.so            
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libutils.so            
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libutils.so              
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libutils.so        
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libutils.so            
warning: section .plt not found in .gnu_debugdata for /system/lib64/libutils.so                 
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libutils.so         
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libutils.so          
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libutils.so          
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libutils.so             
warning: section .got not found in .gnu_debugdata for /system/lib64/libutils.so                 
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libutils.so             
warning: section .data not found in .gnu_debugdata for /system/lib64/libutils.so                
warning: section .bss not found in .gnu_debugdata for /system/lib64/libutils.so                 
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libunwindstack.so                                            
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libunwindstack.so                                             
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libunwindstack.so        
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libunwindstack.so   
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libunwindstack.so 
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libunwindstack.so      
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libunwindstack.so        
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libunwindstack.so      
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libunwindstack.so      
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libunwindstack.so      
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libunwindstack.so        
warning: section .gcc_except_table not found in .gnu_debugdata for /system/lib64/libunwindstack.so                                              
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libunwindstack.so  
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libunwindstack.so      
warning: section .plt not found in .gnu_debugdata for /system/lib64/libunwindstack.so           
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libunwindstack.so   
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libunwindstack.so    
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libunwindstack.so    
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libunwindstack.so       
warning: section .got not found in .gnu_debugdata for /system/lib64/libunwindstack.so           
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libunwindstack.so       
warning: section .data not found in .gnu_debugdata for /system/lib64/libunwindstack.so          
warning: section .bss not found in .gnu_debugdata for /system/lib64/libunwindstack.so           
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libc++.so    
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libc++.so     
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libc++.so                
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libc++.so           
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libc++.so         
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libc++.so              
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libc++.so                
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libc++.so              
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libc++.so              
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libc++.so              
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libc++.so               
warning: section .gcc_except_table not found in .gnu_debugdata for /system/lib64/libc++.so      
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libc++.so          
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libc++.so              
warning: section .plt not found in .gnu_debugdata for /system/lib64/libc++.so                   
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libc++.so           
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libc++.so            
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libc++.so            
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libc++.so               
warning: section .got not found in .gnu_debugdata for /system/lib64/libc++.so                   
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libc++.so               
warning: section .data not found in .gnu_debugdata for /system/lib64/libc++.so                  
warning: section .bss not found in .gnu_debugdata for /system/lib64/libc++.so                   
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libcutils.so 
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libcutils.so  
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libcutils.so             
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libcutils.so        
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libcutils.so      
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libcutils.so           
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libcutils.so             
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libcutils.so           
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libcutils.so           
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libcutils.so           
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libcutils.so             
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libcutils.so       
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libcutils.so           
warning: section .plt not found in .gnu_debugdata for /system/lib64/libcutils.so                
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libcutils.so        
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libcutils.so         
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libcutils.so         
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libcutils.so            
warning: section .got not found in .gnu_debugdata for /system/lib64/libcutils.so                
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libcutils.so            
warning: section .data not found in .gnu_debugdata for /system/lib64/libcutils.so               
warning: section .bss not found in .gnu_debugdata for /system/lib64/libcutils.so                
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/liblog.so    
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/liblog.so     
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/liblog.so                
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/liblog.so           
warning: section .gnu.version_d not found in .gnu_debugdata for /system/lib64/liblog.so         
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/liblog.so         
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/liblog.so              
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/liblog.so                
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/liblog.so              
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/liblog.so              
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/liblog.so              
warning: section .rodata not found in .gnu_debugdata for /system/lib64/liblog.so                
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/liblog.so          
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/liblog.so              
warning: section .plt not found in .gnu_debugdata for /system/lib64/liblog.so                   
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/liblog.so           
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/liblog.so            
warning: section .init_array not found in .gnu_debugdata for /system/lib64/liblog.so            
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/liblog.so               
warning: section .got not found in .gnu_debugdata for /system/lib64/liblog.so                   
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/liblog.so               
warning: section .data not found in .gnu_debugdata for /system/lib64/liblog.so                  
warning: section .bss not found in .gnu_debugdata for /system/lib64/liblog.so                   
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libvndksupport.so                                           
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libvndksupport.so                                            
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libvndksupport.so        
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libvndksupport.so   
warning: section .gnu.version_d not found in .gnu_debugdata for /system/lib64/libvndksupport.so 
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libvndksupport.so 
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libvndksupport.so      
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libvndksupport.so        
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libvndksupport.so      
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libvndksupport.so      
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libvndksupport.so        
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libvndksupport.so  
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libvndksupport.so      
warning: section .plt not found in .gnu_debugdata for /system/lib64/libvndksupport.so           
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libvndksupport.so   
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libvndksupport.so    
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libvndksupport.so       
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libvndksupport.so       
warning: section .bss not found in .gnu_debugdata for /system/lib64/libvndksupport.so           
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libbase.so   
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libbase.so    
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libbase.so               
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libbase.so          
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libbase.so        
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libbase.so             
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libbase.so               
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libbase.so             
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libbase.so             
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libbase.so             
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libbase.so               
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libbase.so         
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libbase.so             
warning: section .plt not found in .gnu_debugdata for /system/lib64/libbase.so                  
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libbase.so          
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libbase.so           
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libbase.so           
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libbase.so              
warning: section .got not found in .gnu_debugdata for /system/lib64/libbase.so                  
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libbase.so              
warning: section .data not found in .gnu_debugdata for /system/lib64/libbase.so                 
warning: section .bss not found in .gnu_debugdata for /system/lib64/libbase.so                  
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/liblzma.so   
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/liblzma.so    
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/liblzma.so               
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/liblzma.so          
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/liblzma.so        
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/liblzma.so             
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/liblzma.so               
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/liblzma.so             
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/liblzma.so             
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/liblzma.so             
warning: section .rodata not found in .gnu_debugdata for /system/lib64/liblzma.so               
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/liblzma.so         
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/liblzma.so             
warning: section .plt not found in .gnu_debugdata for /system/lib64/liblzma.so                  
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/liblzma.so          
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/liblzma.so           
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/liblzma.so              
warning: section .got not found in .gnu_debugdata for /system/lib64/liblzma.so                  
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/liblzma.so              
warning: section .bss not found in .gnu_debugdata for /system/lib64/liblzma.so                  
warning: section .note.gnu.build-id not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                     
warning: section .dynsym not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                                
warning: section .gnu.version not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                           
warning: section .gnu.version_d not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                         
warning: section .gnu.hash not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                              
warning: section .dynstr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                                
warning: section .rela.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                              
warning: section .eh_frame_hdr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                          
warning: section .eh_frame not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                              
warning: section .plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                                   
warning: section .dynamic not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                               
warning: section .got.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so                               
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libnetd_client.so                                            
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libnetd_client.so                                             
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libnetd_client.so        
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libnetd_client.so   
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libnetd_client.so 
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libnetd_client.so      
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libnetd_client.so        
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libnetd_client.so      
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libnetd_client.so      
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libnetd_client.so      
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libnetd_client.so        
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libnetd_client.so  
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libnetd_client.so      
warning: section .plt not found in .gnu_debugdata for /system/lib64/libnetd_client.so           
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libnetd_client.so   
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libnetd_client.so    
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libnetd_client.so       
warning: section .got not found in .gnu_debugdata for /system/lib64/libnetd_client.so           
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libnetd_client.so       
warning: section .data not found in .gnu_debugdata for /system/lib64/libnetd_client.so          
warning: section .bss not found in .gnu_debugdata for /system/lib64/libnetd_client.so           
Log start                                       
main: build = 1 (11dc109)                       
main: built with clang version 17.0.2 for aarch64-unknown-linux-android24                       
main: seed  = 1697346575                        
llama_model_loader: loaded meta data with 19 key-value pairs and 291 tensors from /sdcard/Download/llama-2-7b-chat.Q3_K_S.gguf (version GGUF V2 (latest))                                       
llama_model_loader: - tensor    0:                token_embd.weight q3_K     [  4096, 32000,     1,     1 ]                                     
llama_model_loader: - tensor    1:           blk.0.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor    2:            blk.0.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor    3:            blk.0.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor    4:              blk.0.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor    5:            blk.0.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor    6:              blk.0.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor    7:         blk.0.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor    8:              blk.0.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor    9:              blk.0.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   10:           blk.1.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   11:            blk.1.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   12:            blk.1.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   13:              blk.1.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   14:            blk.1.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   15:              blk.1.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   16:         blk.1.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   17:              blk.1.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   18:              blk.1.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   19:          blk.10.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   20:           blk.10.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   21:           blk.10.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   22:             blk.10.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   23:           blk.10.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   24:             blk.10.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   25:        blk.10.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   26:             blk.10.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   27:             blk.10.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   28:          blk.11.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   29:           blk.11.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   30:           blk.11.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   31:             blk.11.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   32:           blk.11.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   33:             blk.11.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   34:        blk.11.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   35:             blk.11.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   36:             blk.11.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   37:          blk.12.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   38:           blk.12.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   39:           blk.12.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   40:             blk.12.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   41:           blk.12.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   42:             blk.12.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   43:        blk.12.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   44:             blk.12.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   45:             blk.12.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   46:          blk.13.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   47:           blk.13.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   48:           blk.13.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   49:             blk.13.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   50:           blk.13.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   51:             blk.13.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   52:        blk.13.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   53:             blk.13.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   54:             blk.13.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   55:          blk.14.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   56:           blk.14.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   57:           blk.14.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   58:             blk.14.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   59:           blk.14.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   60:             blk.14.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   61:        blk.14.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   62:             blk.14.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   63:             blk.14.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   64:          blk.15.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   65:           blk.15.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   66:           blk.15.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   67:             blk.15.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   68:           blk.15.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   69:             blk.15.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   70:        blk.15.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   71:             blk.15.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   72:             blk.15.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   73:          blk.16.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   74:           blk.16.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   75:           blk.16.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   76:             blk.16.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   77:           blk.16.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   78:             blk.16.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   79:        blk.16.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   80:             blk.16.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   81:             blk.16.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   82:          blk.17.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   83:           blk.17.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   84:           blk.17.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   85:             blk.17.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   86:           blk.17.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   87:             blk.17.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   88:        blk.17.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   89:             blk.17.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   90:             blk.17.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   91:          blk.18.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   92:           blk.18.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   93:           blk.18.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   94:             blk.18.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor   95:           blk.18.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor   96:             blk.18.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   97:        blk.18.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   98:             blk.18.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor   99:             blk.18.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  100:          blk.19.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  101:           blk.19.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  102:           blk.19.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  103:             blk.19.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  104:           blk.19.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  105:             blk.19.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  106:        blk.19.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  107:             blk.19.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  108:             blk.19.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  109:           blk.2.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  110:            blk.2.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  111:            blk.2.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  112:              blk.2.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  113:            blk.2.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  114:              blk.2.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  115:         blk.2.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  116:              blk.2.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  117:              blk.2.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  118:          blk.20.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  119:           blk.20.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  120:           blk.20.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  121:             blk.20.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  122:           blk.20.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  123:             blk.20.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  124:        blk.20.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  125:             blk.20.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  126:             blk.20.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  127:          blk.21.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  128:           blk.21.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  129:           blk.21.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  130:             blk.21.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  131:           blk.21.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  132:             blk.21.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  133:        blk.21.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  134:             blk.21.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  135:             blk.21.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  136:          blk.22.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  137:           blk.22.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  138:           blk.22.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  139:             blk.22.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  140:           blk.22.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  141:             blk.22.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  142:        blk.22.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  143:             blk.22.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  144:             blk.22.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  145:          blk.23.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  146:           blk.23.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  147:           blk.23.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  148:             blk.23.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  149:           blk.23.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  150:             blk.23.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  151:        blk.23.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  152:             blk.23.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  153:             blk.23.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  154:           blk.3.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  155:            blk.3.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  156:            blk.3.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  157:              blk.3.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  158:            blk.3.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  159:              blk.3.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  160:         blk.3.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  161:              blk.3.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  162:              blk.3.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  163:           blk.4.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  164:            blk.4.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  165:            blk.4.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  166:              blk.4.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  167:            blk.4.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  168:              blk.4.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  169:         blk.4.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  170:              blk.4.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  171:              blk.4.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  172:           blk.5.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  173:            blk.5.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  174:            blk.5.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  175:              blk.5.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  176:            blk.5.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  177:              blk.5.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  178:         blk.5.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  179:              blk.5.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  180:              blk.5.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  181:           blk.6.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  182:            blk.6.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  183:            blk.6.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  184:              blk.6.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  185:            blk.6.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  186:              blk.6.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  187:         blk.6.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  188:              blk.6.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  189:              blk.6.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  190:           blk.7.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  191:            blk.7.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  192:            blk.7.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  193:              blk.7.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  194:            blk.7.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  195:              blk.7.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  196:         blk.7.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  197:              blk.7.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  198:              blk.7.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  199:           blk.8.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  200:            blk.8.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  201:            blk.8.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  202:              blk.8.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  203:            blk.8.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  204:              blk.8.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  205:         blk.8.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  206:              blk.8.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  207:              blk.8.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  208:           blk.9.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  209:            blk.9.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  210:            blk.9.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  211:              blk.9.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  212:            blk.9.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  213:              blk.9.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  214:         blk.9.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  215:              blk.9.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  216:              blk.9.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  217:                    output.weight q6_K     [  4096, 32000,     1,     1 ]                                     
llama_model_loader: - tensor  218:          blk.24.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  219:           blk.24.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  220:           blk.24.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  221:             blk.24.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  222:           blk.24.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  223:             blk.24.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  224:        blk.24.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  225:             blk.24.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  226:             blk.24.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  227:          blk.25.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  228:           blk.25.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  229:           blk.25.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  230:             blk.25.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  231:           blk.25.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  232:             blk.25.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  233:        blk.25.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  234:             blk.25.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  235:             blk.25.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  236:          blk.26.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  237:           blk.26.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  238:           blk.26.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  239:             blk.26.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  240:           blk.26.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  241:             blk.26.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  242:        blk.26.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  243:             blk.26.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  244:             blk.26.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  245:          blk.27.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  246:           blk.27.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  247:           blk.27.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  248:             blk.27.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  249:           blk.27.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  250:             blk.27.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  251:        blk.27.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  252:             blk.27.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  253:             blk.27.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  254:          blk.28.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  255:           blk.28.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  256:           blk.28.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  257:             blk.28.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  258:           blk.28.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  259:             blk.28.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  260:        blk.28.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  261:             blk.28.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  262:             blk.28.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  263:          blk.29.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  264:           blk.29.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  265:           blk.29.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  266:             blk.29.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  267:           blk.29.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  268:             blk.29.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  269:        blk.29.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  270:             blk.29.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  271:             blk.29.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  272:          blk.30.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  273:           blk.30.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  274:           blk.30.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  275:             blk.30.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  276:           blk.30.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  277:             blk.30.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  278:        blk.30.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  279:             blk.30.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  280:             blk.30.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  281:          blk.31.attn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  282:           blk.31.ffn_down.weight q3_K     [ 11008,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  283:           blk.31.ffn_gate.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  284:             blk.31.ffn_up.weight q3_K     [  4096, 11008,     1,     1 ]                                     
llama_model_loader: - tensor  285:           blk.31.ffn_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - tensor  286:             blk.31.attn_k.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  287:        blk.31.attn_output.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  288:             blk.31.attn_q.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  289:             blk.31.attn_v.weight q3_K     [  4096,  4096,     1,     1 ]                                     
llama_model_loader: - tensor  290:               output_norm.weight f32      [  4096,     1,     1,     1 ]                                     
llama_model_loader: - kv   0:                       general.architecture str                    
llama_model_loader: - kv   1:                               general.name str                    
llama_model_loader: - kv   2:                       llama.context_length u32                    
llama_model_loader: - kv   3:                     llama.embedding_length u32                    
llama_model_loader: - kv   4:                          llama.block_count u32                    
llama_model_loader: - kv   5:                  llama.feed_forward_length u32                    
llama_model_loader: - kv   6:                 llama.rope.dimension_count u32                    
llama_model_loader: - kv   7:                 llama.attention.head_count u32                    
llama_model_loader: - kv   8:              llama.attention.head_count_kv u32                    
llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32                    
llama_model_loader: - kv  10:                          general.file_type u32                    
llama_model_loader: - kv  11:                       tokenizer.ggml.model str                    
llama_model_loader: - kv  12:                      tokenizer.ggml.tokens arr                    
llama_model_loader: - kv  13:                      tokenizer.ggml.scores arr                    
llama_model_loader: - kv  14:                  tokenizer.ggml.token_type arr                    
llama_model_loader: - kv  15:                tokenizer.ggml.bos_token_id u32                    
llama_model_loader: - kv  16:                tokenizer.ggml.eos_token_id u32                    
llama_model_loader: - kv  17:            tokenizer.ggml.unknown_token_id u32                    
llama_model_loader: - kv  18:               general.quantization_version u32                    
llama_model_loader: - type  f32:   65 tensors   
llama_model_loader: - type q3_K:  225 tensors   
llama_model_loader: - type q6_K:    1 tensors   
llm_load_print_meta: format           = GGUF V2 (latest)                                        
llm_load_print_meta: arch             = llama   
llm_load_print_meta: vocab type       = SPM     
llm_load_print_meta: n_vocab          = 32000   
llm_load_print_meta: n_merges         = 0       
llm_load_print_meta: n_ctx_train      = 4096    
llm_load_print_meta: n_embd           = 4096    
llm_load_print_meta: n_head           = 32      
llm_load_print_meta: n_head_kv        = 32      
llm_load_print_meta: n_layer          = 32      
llm_load_print_meta: n_rot            = 128     
llm_load_print_meta: n_gqa            = 1       
llm_load_print_meta: f_norm_eps       = 0.0e+00 
llm_load_print_meta: f_norm_rms_eps   = 1.0e-06 
llm_load_print_meta: f_clamp_kqv      = 0.0e+00 
llm_load_print_meta: f_max_alibi_bias = 0.0e+00 
llm_load_print_meta: n_ff             = 11008   
llm_load_print_meta: freq_base_train  = 10000.0 
llm_load_print_meta: freq_scale_train = 1       
llm_load_print_meta: model type       = 7B      
llm_load_print_meta: model ftype      = mostly Q3_K - Small                                     
llm_load_print_meta: model params     = 6.74 B  
llm_load_print_meta: model size       = 2.75 GiB (3.50 BPW)                                     
llm_load_print_meta: general.name   = LLaMA v2  
llm_load_print_meta: BOS token = 1 '<s>'        
llm_load_print_meta: EOS token = 2 '</s>'       
llm_load_print_meta: UNK token = 0 '<unk>'      
llm_load_print_meta: LF token  = 13 '<0x0A>'    
llm_load_tensors: ggml ctx size =    0.10 MB    
llm_load_tensors: mem required  = 2811.11 MB    
.................................................................................................                                               
llama_new_context_with_model: n_ctx      = 2048 
llama_new_context_with_model: freq_base  = 10000.0                                              
llama_new_context_with_model: freq_scale = 1    
llama_new_context_with_model: kv self size  = 1024.00 MB                                        
llama_new_context_with_model: compute buffer total size = 9.17 MB                               
[New Thread 0x137d (LWP 4989)]                  
[New Thread 0x137e (LWP 4990)]                                                                  
[Thread 0x137e (LWP 4990) exited]               
[Thread 0x137d (LWP 4989) exited]               
system_info: n_threads = 3 / 8 | AVX = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 |                  
main: interactive mode on.                      
Reverse prompt: '### Instruction:                                                               '                                               sampling: repeat_last_n = 64, repeat_penalty = 1.100000, presence_penalty = 0.000000, frequency_penalty = 0.000000, top_k = 40, tfs_z = 1.000000, top_p = 0.950000, typical_p = 1.000000, temp = 0.800000, mirostat = 0, mirostat_lr = 0.100000, mirostat_ent = 5.000000                        
generate: n_ctx = 2048, n_batch = 10, n_predict = -1, n_keep = 1

== Running in interactive mode. ==               
- Press Ctrl+C to interject at any time.        
- Press Return to return control to LLaMa.      
- To return control without starting a new line, end your input with '/'.                       
- If you want to submit another line, end your input with '\'.                                                                                 
[New Thread 0x1380 (LWP 4992)]                  
[New Thread 0x1381 (LWP 4993)]                                                                  
> [Thread 0x1381 (LWP 4993) exited]             
[Thread 0x1380 (LWP 4992) exited]                                                                
Hinweis[New Thread 0x138b (LWP 5003)]          
[New Thread 0x138c (LWP 5004)]                 
[Thread 0x138c (LWP 5004) exited]             
[Thread 0x138b (LWP 5003) exited]               
,[New Thread 0x138e (LWP 5006)]                 
[New Thread 0x138f (LWP 5007)]                  
[Thread 0x138f (LWP 5007) exited]               
[Thread 0x138e (LWP 5006) exited]                
I[New Thread 0x1398 (LWP 5016)]                
[New Thread 0x1399 (LWP 5017)]                  
[Thread 0x1399 (LWP 5017) exited]               
[Thread 0x1398 (LWP 5016) exited]                
am[New Thread 0x139b (LWP 5019)]               
[New Thread 0x139c (LWP 5020)]                  
[Thread 0x139c (LWP 5020) exited]               
[Thread 0x139b (LWP 5019) exited]                
not[New Thread 0x139e (LWP 5022)]              
[New Thread 0x139f (LWP 5023)]                  
[Thread 0x139f (LWP 5023) exited]               
[Thread 0x139e (LWP 5022) exited]                
able[New Thread 0x13a1 (LWP 5025)]             
[New Thread 0x13a2 (LWP 5026)]                  
[Thread 0x13a2 (LWP 5026) exited]               
[Thread 0x13a1 (LWP 5025) exited]                
to[New Thread 0x13a4 (LWP 5028)]               
[New Thread 0x13a7 (LWP 5031)]                  
[Thread 0x13a7 (LWP 5031) exited]               
[Thread 0x13a4 (LWP 5028) exited]                
answer[New Thread 0x13a9 (LWP 5033)]           
[New Thread 0x13aa (LWP 5034)]                  
[Thread 0x13aa (LWP 5034) exited]               
[Thread 0x13a9 (LWP 5033) exited]                
all[New Thread 0x13ad (LWP 5037)]              
[New Thread 0x13ae (LWP 5038)]                  
[Thread 0x13ae (LWP 5038) exited]               
[Thread 0x13ad (LWP 5037) exited]                
of[New Thread 0x13b4 (LWP 5044)]               
[New Thread 0x13d1 (LWP 5073)]                  
[Thread 0x13d1 (LWP 5073) exited]               
[Thread 0x13b4 (LWP 5044) exited]                
the[New Thread 0x13de (LWP 5086)]              
[New Thread 0x13df (LWP 5087)]                  
[Thread 0x13df (LWP 5087) exited]               
[Thread 0x13de (LWP 5086) exited]                
questions[New Thread 0x13e1 (LWP 5089)]        
[New Thread 0x13e2 (LWP 5090)]                  
[Thread 0x13e2 (LWP 5090) exited]               
[Thread 0x13e1 (LWP 5089) exited]                
in[New Thread 0x13e6 (LWP 5094)]               
[New Thread 0x13e7 (LWP 5095)]                  
[Thread 0x13e7 (LWP 5095) exited]               
[Thread 0x13e6 (LWP 5094) exited]                
the[New Thread 0x13e9 (LWP 5097)]              
[New Thread 0x13ea (LWP 5098)]                  
[Thread 0x13ea (LWP 5098) exited]               
[Thread 0x13e9 (LWP 5097) exited]                
qu[New Thread 0x13ec (LWP 5100)]               
[New Thread 0x13ed (LWP 5101)]                  
[Thread 0x13ed (LWP 5101) exited]               
[Thread 0x13ec (LWP 5100) exited]               
iz[New Thread 0x13ef (LWP 5103)]                
[New Thread 0x13f0 (LWP 5104)]                  
[Thread 0x13f0 (LWP 5104) exited]               
[Thread 0x13ef (LWP 5103) exited]                
as[New Thread 0x13f2 (LWP 5106)]               
[New Thread 0x13f3 (LWP 5107)]                  
[Thread 0x13f3 (LWP 5107) exited]               
[Thread 0x13f2 (LWP 5106) exited]                
some[New Thread 0x13f5 (LWP 5109)]             
[New Thread 0x13f6 (LWP 5110)]                  
[Thread 0x13f6 (LWP 5110) exited]               
[Thread 0x13f5 (LWP 5109) exited]                
of[New Thread 0x13f8 (LWP 5112)]               
[New Thread 0x13f9 (LWP 5113)]                  
[Thread 0x13f9 (LWP 5113) exited]               
[Thread 0x13f8 (LWP 5112) exited]                
the[New Thread 0x13fb (LWP 5115)]              
[New Thread 0x13fc (LWP 5116)]                  
[Thread 0x13fc (LWP 5116) exited]               
[Thread 0x13fb (LWP 5115) exited]                
information[New Thread 0x13fe (LWP 5118)]      
[New Thread 0x13ff (LWP 5119)]                  
[Thread 0x13ff (LWP 5119) exited]               
[Thread 0x13fe (LWP 5118) exited]                
is[New Thread 0x1401 (LWP 5121)]               
[New Thread 0x1402 (LWP 5122)]                  
[Thread 0x1402 (LWP 5122) exited]               
[Thread 0x1401 (LWP 5121) exited]                
not[New Thread 0x1404 (LWP 5124)]              
[New Thread 0x1405 (LWP 5125)]                  
[Thread 0x1405 (LWP 5125) exited]               
[Thread 0x1404 (LWP 5124) exited]                
available[New Thread 0x1407 (LWP 5127)]        
[New Thread 0x1408 (LWP 5128)]                  
[Thread 0x1408 (LWP 5128) exited]               
[Thread 0x1407 (LWP 5127) exited]                
to[New Thread 0x140a (LWP 5130)]               
[New Thread 0x140b (LWP 5131)]                  
[Thread 0x140b (LWP 5131) exited]                
me[Thread 0x140a (LWP 5130) exited]            
[New Thread 0x140d (LWP 5133)]                  
[New Thread 0x140e (LWP 5134)]                  
[Thread 0x140e (LWP 5134) exited]               
[Thread 0x140d (LWP 5133) exited]               
.[New Thread 0x1410 (LWP 5136)]                 
[New Thread 0x1411 (LWP 5137)]                  
[Thread 0x1411 (LWP 5137) exited]                
nobody[Thread 0x1410 (LWP 5136) exited]        
[New Thread 0x1413 (LWP 5139)]                  
[New Thread 0x1414 (LWP 5140)]                  
[Thread 0x1414 (LWP 5140) exited]               
[Thread 0x1413 (LWP 5139) exited]                
has[New Thread 0x1416 (LWP 5142)]              
[New Thread 0x1417 (LWP 5143)]                  
[Thread 0x1417 (LWP 5143) exited]               
[Thread 0x1416 (LWP 5142) exited]                
been[New Thread 0x1419 (LWP 5145)]             
[New Thread 0x141a (LWP 5146)]                  
[Thread 0x141a (LWP 5146) exited]               
[Thread 0x1419 (LWP 5145) exited]                
able[New Thread 0x141c (LWP 5148)]             
[New Thread 0x141d (LWP 5149)]                  
[Thread 0x141d (LWP 5149) exited]               
[Thread 0x141c (LWP 5148) exited]                
to[New Thread 0x141f (LWP 5151)]               
[New Thread 0x1420 (LWP 5152)]                  
[Thread 0x1420 (LWP 5152) exited]               
[Thread 0x141f (LWP 5151) exited]                
create[New Thread 0x1422 (LWP 5154)]           
[New Thread 0x1423 (LWP 5155)]                  
[Thread 0x1423 (LWP 5155) exited]               
[Thread 0x1422 (LWP 5154) exited]                
a[New Thread 0x1425 (LWP 5157)]                
[New Thread 0x1426 (LWP 5158)]                  
[Thread 0x1426 (LWP 5158) exited]               
[Thread 0x1425 (LWP 5157) exited]                
vi[New Thread 0x142a (LWP 5162)]               
[New Thread 0x142b (LWP 5163)]                  
[Thread 0x142b (LWP 5163) exited]               
[Thread 0x142a (LWP 5162) exited]               
able[New Thread 0x1431 (LWP 5169)]              
[New Thread 0x1432 (LWP 5170)]                  
[Thread 0x1432 (LWP 5170) exited]               
[Thread 0x1431 (LWP 5169) exited]                
A[New Thread 0x1434 (LWP 5172)]                
[New Thread 0x1435 (LWP 5173)]                  
[Thread 0x1435 (LWP 5173) exited]               
[Thread 0x1434 (LWP 5172) exited]               
I[New Thread 0x1437 (LWP 5175)]                 
[New Thread 0x1439 (LWP 5177)]                  
[Thread 0x1439 (LWP 5177) exited]               
[Thread 0x1437 (LWP 5175) exited]                
language[New Thread 0x143c (LWP 5180)]         
[New Thread 0x143d (LWP 5181)]                  
[Thread 0x143d (LWP 5181) exited]               
[Thread 0x143c (LWP 5180) exited]                
model[New Thread 0x143f (LWP 5183)]            
[New Thread 0x1440 (LWP 5184)]                  
[Thread 0x1440 (LWP 5184) exited]               
[Thread 0x143f (LWP 5183) exited]                
that[New Thread 0x1445 (LWP 5189)]             
[New Thread 0x1447 (LWP 5191)]                  
[Thread 0x1447 (LWP 5191) exited]                
can[Thread 0x1445 (LWP 5189) exited]           
[New Thread 0x144a (LWP 5194)]                  
[New Thread 0x144c (LWP 5196)]                  
[Thread 0x144c (LWP 5196) exited]               
[Thread 0x144a (LWP 5194) exited]                
understand[New Thread 0x1450 (LWP 5200)]       
[New Thread 0x1451 (LWP 5201)]                  
[Thread 0x1451 (LWP 5201) exited]               
[Thread 0x1450 (LWP 5200) exited]                
and[New Thread 0x1453 (LWP 5203)]              
[New Thread 0x1454 (LWP 5204)]                  
[Thread 0x1454 (LWP 5204) exited]               
[Thread 0x1453 (LWP 5203) exited]                
respond[New Thread 0x1457 (LWP 5207)]          
[New Thread 0x1459 (LWP 5209)]                  
[Thread 0x1459 (LWP 5209) exited]               
[Thread 0x1457 (LWP 5207) exited]                
to[New Thread 0x145b (LWP 5211)]               
[New Thread 0x145c (LWP 5212)]                  
[Thread 0x145c (LWP 5212) exited]               
[Thread 0x145b (LWP 5211) exited]                
complex[New Thread 0x145f (LWP 5215)]          
[New Thread 0x1460 (LWP 5216)]                  
[Thread 0x1460 (LWP 5216) exited]               
[Thread 0x145f (LWP 5215) exited]                
queries[New Thread 0x1464 (LWP 5220)]          
[New Thread 0x1465 (LWP 5221)]                  
[Thread 0x1465 (LWP 5221) exited]               
[Thread 0x1464 (LWP 5220) exited]                
like[New Thread 0x1467 (LWP 5223)]             
[New Thread 0x1468 (LWP 5224)]                  
[Thread 0x1468 (LWP 5224) exited]               
[Thread 0x1467 (LWP 5223) exited]                
a[New Thread 0x146a (LWP 5226)]                
[New Thread 0x146b (LWP 5227)]                  
[Thread 0x146b (LWP 5227) exited]               
[Thread 0x146a (LWP 5226) exited]                
human[New Thread 0x146e (LWP 5230)]            
[New Thread 0x146f (LWP 5231)]                  
[Thread 0x146f (LWP 5231) exited]               
[Thread 0x146e (LWP 5230) exited]                
would[New Thread 0x1472 (LWP 5234)]            
[New Thread 0x1474 (LWP 5236)]                  
[Thread 0x1474 (LWP 5236) exited]               
[Thread 0x1472 (LWP 5234) exited]               
.[New Thread 0x1476 (LWP 5238)]                 
[New Thread 0x1477 (LWP 5239)]                  
[Thread 0x1477 (LWP 5239) exited]               
[Thread 0x1476 (LWP 5238) exited]                                                               
[New Thread 0x1479 (LWP 5241)]                  
[New Thread 0x147a (LWP 5242)]                  
[Thread 0x147a (LWP 5242) exited]               
[Thread 0x1479 (LWP 5241) exited]               
However[New Thread 0x147c (LWP 5244)]           
[New Thread 0x147d (LWP 5245)]                  
[Thread 0x147d (LWP 5245) exited]               
[Thread 0x147c (LWP 5244) exited]               
,[New Thread 0x147f (LWP 5247)]                 
[New Thread 0x1480 (LWP 5248)]                  
[Thread 0x1480 (LWP 5248) exited]               
[Thread 0x147f (LWP 5247) exited]                
I[New Thread 0x1485 (LWP 5253)]                
[New Thread 0x1486 (LWP 5254)]                  
[Thread 0x1486 (LWP 5254) exited]               
[Thread 0x1485 (LWP 5253) exited]                
can[New Thread 0x1498 (LWP 5272)]              
[New Thread 0x1499 (LWP 5273)]                  
[Thread 0x1499 (LWP 5273) exited]               
[Thread 0x1498 (LWP 5272) exited]                
try[New Thread 0x14ad (LWP 5293)]              
[New Thread 0x14ae (LWP 5294)]                  
[Thread 0x14ae (LWP 5294) exited]               
[Thread 0x14ad (LWP 5293) exited]                
to[New Thread 0x14f7 (LWP 5367)]               
[New Thread 0x14f8 (LWP 5368)]                  
[Thread 0x14f8 (LWP 5368) exited]               
[Thread 0x14f7 (LWP 5367) exited]                
help[New Thread 0x1520 (LWP 5408)]             
[New Thread 0x1521 (LWP 5409)]                  
[Thread 0x1521 (LWP 5409) exited]               
[Thread 0x1520 (LWP 5408) exited]                
you[New Thread 0x1525 (LWP 5413)]              
[New Thread 0x1526 (LWP 5414)]                  
[Thread 0x1526 (LWP 5414) exited]               
[Thread 0x1525 (LWP 5413) exited]                
with[New Thread 0x1545 (LWP 5445)]             
[New Thread 0x1546 (LWP 5446)]                  
[Thread 0x1546 (LWP 5446) exited]               
[Thread 0x1545 (LWP 5445) exited]                
any[New Thread 0x15a1 (LWP 5537)]              
[New Thread 0x15a2 (LWP 5538)]                  
[Thread 0x15a2 (LWP 5538) exited]               
[Thread 0x15a1 (LWP 5537) exited]                
specific[New Thread 0x172f (LWP 5935)]         
[New Thread 0x1730 (LWP 5936)]                  
[Thread 0x1730 (LWP 5936) exited]               
[Thread 0x172f (LWP 5935) exited]                
questions[New Thread 0x1781 (LWP 6017)]        
[New Thread 0x1782 (LWP 6018)]                  
[Thread 0x1782 (LWP 6018) exited]               
[Thread 0x1781 (LWP 6017) exited]                
or[New Thread 0x17a5 (LWP 6053)]               
[New Thread 0x17a7 (LWP 6055)]                  
[Thread 0x17a7 (LWP 6055) exited]               
[Thread 0x17a5 (LWP 6053) exited]                
topics[New Thread 0x17d4 (LWP 6100)]           
[New Thread 0x17d5 (LWP 6101)]                  
[Thread 0x17d5 (LWP 6101) exited]               
[Thread 0x17d4 (LWP 6100) exited]                
you[New Thread 0x17ea (LWP 6122)]              
[New Thread 0x17eb (LWP 6123)]                  
[Thread 0x17eb (LWP 6123) exited]               
[Thread 0x17ea (LWP 6122) exited]                
have[New Thread 0x17f8 (LWP 6136)]             
[New Thread 0x17f9 (LWP 6137)]                  
[Thread 0x17f9 (LWP 6137) exited]               
[Thread 0x17f8 (LWP 6136) exited]                
in[New Thread 0x180d (LWP 6157)]               
[New Thread 0x180e (LWP 6158)]                  
[Thread 0x180e (LWP 6158) exited]               
[Thread 0x180d (LWP 6157) exited]                
mind[New Thread 0x1816 (LWP 6166)]             
[New Thread 0x1817 (LWP 6167)]                  
[Thread 0x1817 (LWP 6167) exited]               
[Thread 0x1816 (LWP 6166) exited]               
.[New Thread 0x1820 (LWP 6176)]                 
[New Thread 0x1821 (LWP 6177)]                  
[Thread 0x1821 (LWP 6177) exited]               
[Thread 0x1820 (LWP 6176) exited]                
Just[New Thread 0x182f (LWP 6191)]             
[New Thread 0x1830 (LWP 6192)]                  
[Thread 0x1830 (LWP 6192) exited]               
[Thread 0x182f (LWP 6191) exited]                
let[New Thread 0x185e (LWP 6238)]              
[New Thread 0x1861 (LWP 6241)]                  
[Thread 0x1861 (LWP 6241) exited]               
[Thread 0x185e (LWP 6238) exited]                
me[New Thread 0x1877 (LWP 6263)]               
[New Thread 0x1879 (LWP 6265)]                  
[Thread 0x1879 (LWP 6265) exited]               
[Thread 0x1877 (LWP 6263) exited]                
know[New Thread 0x1883 (LWP 6275)]             
[New Thread 0x1884 (LWP 6276)]                  
[Thread 0x1884 (LWP 6276) exited]               
[Thread 0x1883 (LWP 6275) exited]                
what[New Thread 0x188a (LWP 6282)]             
[New Thread 0x188b (LWP 6283)]                  
[Thread 0x188b (LWP 6283) exited]               
[Thread 0x188a (LWP 6282) exited]               
'[New Thread 0x188d (LWP 6285)]                 
[New Thread 0x188e (LWP 6286)]                  
[Thread 0x188e (LWP 6286) exited]               
[Thread 0x188d (LWP 6285) exited]               
s[New Thread 0x1893 (LWP 6291)]                 
[New Thread 0x1894 (LWP 6292)]                  
[Thread 0x1894 (LWP 6292) exited]               
[Thread 0x1893 (LWP 6291) exited]                
on[New Thread 0x18a8 (LWP 6312)]               
[New Thread 0x18a9 (LWP 6313)]                  
[Thread 0x18a9 (LWP 6313) exited]               
[Thread 0x18a8 (LWP 6312) exited]                
your[New Thread 0x18b4 (LWP 6324)]             
[New Thread 0x18b5 (LWP 6325)]                  
[Thread 0x18b5 (LWP 6325) exited]               
[Thread 0x18b4 (LWP 6324) exited]                
mind[New Thread 0x18c3 (LWP 6339)]             
[New Thread 0x18c4 (LWP 6340)]                  
[Thread 0x18c4 (LWP 6340) exited]               
[Thread 0x18c3 (LWP 6339) exited]               
,[New Thread 0x18c7 (LWP 6343)]                 
[New Thread 0x18c8 (LWP 6344)]                  
[Thread 0x18c8 (LWP 6344) exited]               
[Thread 0x18c7 (LWP 6343) exited]                
and[New Thread 0x18cb (LWP 6347)]              
[New Thread 0x18cc (LWP 6348)]                  
[Thread 0x18cc (LWP 6348) exited]               
[Thread 0x18cb (LWP 6347) exited]                
I[New Thread 0x18d1 (LWP 6353)]                
[New Thread 0x18d2 (LWP 6354)]                  
[Thread 0x18d2 (LWP 6354) exited]               
[Thread 0x18d1 (LWP 6353) exited]               
'[New Thread 0x18d5 (LWP 6357)]                 
[New Thread 0x18d6 (LWP 6358)]                  
[Thread 0x18d6 (LWP 6358) exited]               
[Thread 0x18d5 (LWP 6357) exited]               
ll[New Thread 0x18d9 (LWP 6361)]                
[New Thread 0x18da (LWP 6362)]                  
[Thread 0x18da (LWP 6362) exited]               
[Thread 0x18d9 (LWP 6361) exited]                
do[New Thread 0x18dc (LWP 6364)]               
[New Thread 0x18dd (LWP 6365)]                  
[Thread 0x18dd (LWP 6365) exited]               
[Thread 0x18dc (LWP 6364) exited]                
my[New Thread 0x18df (LWP 6367)]               
[New Thread 0x18e0 (LWP 6368)]                  
[Thread 0x18e0 (LWP 6368) exited]               
[Thread 0x18df (LWP 6367) exited]                
best[New Thread 0x18e4 (LWP 6372)]             
[New Thread 0x18e5 (LWP 6373)]                  
[Thread 0x18e5 (LWP 6373) exited]               
[Thread 0x18e4 (LWP 6372) exited]                
to[New Thread 0x18e9 (LWP 6377)]               
[New Thread 0x18ea (LWP 6378)]                  
[Thread 0x18ea (LWP 6378) exited]               
[Thread 0x18e9 (LWP 6377) exited]                
provide[New Thread 0x18ec (LWP 6380)]          
[New Thread 0x18ed (LWP 6381)]                  
[Thread 0x18ed (LWP 6381) exited]               
[Thread 0x18ec (LWP 6380) exited]                
helpful[New Thread 0x18f2 (LWP 6386)]          
[New Thread 0x18f3 (LWP 6387)]                  
[Thread 0x18f3 (LWP 6387) exited]               
[Thread 0x18f2 (LWP 6386) exited]                
and[New Thread 0x18f8 (LWP 6392)]              
[New Thread 0x18f9 (LWP 6393)]                  
[Thread 0x18f9 (LWP 6393) exited]               
[Thread 0x18f8 (LWP 6392) exited]                
accurate[New Thread 0x18fd (LWP 6397)]         
[New Thread 0x18fe (LWP 6398)]                  
[Thread 0x18fe (LWP 6398) exited]               
[Thread 0x18fd (LWP 6397) exited]                
information[New Thread 0x1901 (LWP 6401)]      
[New Thread 0x1902 (LWP 6402)]                  
[Thread 0x1902 (LWP 6402) exited]               
[Thread 0x1901 (LWP 6401) exited]               
.[New Thread 0x1908 (LWP 6408)]                 
[New Thread 0x1909 (LWP 6409)]                  
[Thread 0x1909 (LWP 6409) exited]               
[Thread 0x1908 (LWP 6408) exited]                                                                                                               
> hi                                            
[New Thread 0x1929 (LWP 6441)]                  
[New Thread 0x192a (LWP 6442)]                  
[Thread 0x192a (LWP 6442) exited]               
[Thread 0x1929 (LWP 6441) exited]               
[New Thread 0x192d (LWP 6445)]                  
[New Thread 0x192e (LWP 6446)]                  
[Thread 0x192e (LWP 6446) exited]               
[New Thread 0x1931 (LWP 6449)]                  
[Thread 0x192d (LWP 6445) exited]               
[New Thread 0x1932 (LWP 6450)]                  
[Thread 0x1932 (LWP 6450) exited]               
[Thread 0x1931 (LWP 6449) exited]               
Hello[New Thread 0x1936 (LWP 6454)]             
[New Thread 0x1937 (LWP 6455)]                  
[Thread 0x1937 (LWP 6455) exited]               
[Thread 0x1936 (LWP 6454) exited]               
![New Thread 0x1939 (LWP 6457)]                 
[New Thread 0x193a (LWP 6458)]                  
[Thread 0x193a (LWP 6458) exited]               
[Thread 0x1939 (LWP 6457) exited]                
It[New Thread 0x193c (LWP 6460)]               
[New Thread 0x193d (LWP 6461)]                  
[Thread 0x193d (LWP 6461) exited]               
[Thread 0x193c (LWP 6460) exited]               
'[New Thread 0x193f (LWP 6463)]                 
[New Thread 0x1940 (LWP 6464)]                  
[Thread 0x1940 (LWP 6464) exited]               
[Thread 0x193f (LWP 6463) exited]               
s[New Thread 0x1942 (LWP 6466)]                 
[New Thread 0x1943 (LWP 6467)]                  
[Thread 0x1943 (LWP 6467) exited]               
[Thread 0x1942 (LWP 6466) exited]                
nice[New Thread 0x1945 (LWP 6469)]             
[New Thread 0x1946 (LWP 6470)]                  
[Thread 0x1946 (LWP 6470) exited]               
[Thread 0x1945 (LWP 6469) exited]                
to[New Thread 0x1948 (LWP 6472)]               
[New Thread 0x1949 (LWP 6473)]                  
[Thread 0x1949 (LWP 6473) exited]               
[Thread 0x1948 (LWP 6472) exited]                
meet[New Thread 0x194b (LWP 6475)]             
[New Thread 0x194c (LWP 6476)]                  
[Thread 0x194c (LWP 6476) exited]               
[Thread 0x194b (LWP 6475) exited]                
you[New Thread 0x194f (LWP 6479)]              
[New Thread 0x1950 (LWP 6480)]                  
[Thread 0x1950 (LWP 6480) exited]               
[Thread 0x194f (LWP 6479) exited]               
.[New Thread 0x1957 (LWP 6487)]                 
[New Thread 0x1958 (LWP 6488)]                                                                  
Thread 1 "main" received signal SIGINT, Interrupt.                                              
0x00000055555fb178 in ggml_graph_compute_thread (data=data@entry=0x7ffffecd60) at /data/data/com.termux/files/home/llama.cpp/ggml.c:18358       18358                   
node_n = atomic_load(&state->shared->node_n);                           
(gdb) bt                                        
#0  0x00000055555fb178 in ggml_graph_compute_thread (data=data@entry=0x7ffffecd60) at /data/data/com.termux/files/home/llama.cpp/ggml.c:18358   
#1  0x00000055555faeec in ggml_graph_compute (cgraph=0xb400007c5bbb7fc0, cplan=<optimized out>) at /data/data/com.termux/files/home/llama.cpp/ggml.c:18903                                      
#2  0x00000055555c83d0 in ggml_graph_compute_helper (buf=..., graph=graph@entry=0xb400007c5bbb7fc0, n_threads=<optimized out>, n_threads@entry=3) at /data/data/com.termux/files/home/llama.cpp/llama.cpp:567                                   
#3  0x00000055555c8e28 in llama_decode_internal (lctx=..., batch=...) at /data/data/com.termux/files/home/llama.cpp/llama.cpp:5868              
#4  0x00000055555c947c in llama_decode (ctx=0x2, batch=...) at /data/data/com.termux/files/home/llama.cpp/llama.cpp:9351                        
#5  0x000000555558cd38 in main (argc=<optimized out>, argv=<optimized out>) at /data/data/com.termux/files/home/llama.cpp/examples/main/main.cpp:602                                            
(gdb) quit                                      
A debugging session is active.                                                                          
Inferior 1 [process 4985] will be killed.                                                                                               
Quit anyway? (y or n) y                         
~/llama.cpp $

theoctopusride commented 12 months ago

I had success on other phones using make -C llama.cpp -j4 and llama.cpp/server -m /sdcard/Download/llama-2-7b-chat.Q3_K_S.gguf

but after re-cloning and re-compiling, when running the stack trace with the llama.cpp/server command, I see:

Starting program: /data/data/com.termux/files/home/llama.cpp/server -m /sdcard/Download/llama-2-7b-chat.Q3_K_S.gguf
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/data/data/com.termux/files/usr/lib/libthread_db.so".
warning: section .note.gnu.build-id not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .dynsym not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .gnu.version not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .gnu.version_d not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .gnu.hash not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .dynstr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .rela.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .eh_frame not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .dynamic not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .got.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .bss not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl.so
warning: section .note.android.ident not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .dynsym not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .gnu.version not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .gnu.version_d not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .gnu.version_r not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .gnu.hash not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .dynstr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .rela.dyn not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .relr.dyn not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .rela.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .rodata not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .eh_frame not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .data.rel.ro not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .fini_array not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .dynamic not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .got not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .got.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .data not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .bss not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libm.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .gnu.version_d not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libmemleak_tracker.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libc_secshared.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libutils.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .gcc_except_table not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libunwindstack.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .gcc_except_table not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libc++.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libcutils.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .gnu.version_d not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .got not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .data not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/liblog.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .gnu.version_d not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libvndksupport.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .init_array not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libbase.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .got not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/liblzma.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .dynsym not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .gnu.version not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .gnu.version_d not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .gnu.hash not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .dynstr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .rela.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .eh_frame not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .dynamic not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .got.plt not found in .gnu_debugdata for /apex/com.android.runtime/lib64/bionic/libdl_android.so
warning: section .note.android.ident not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .note.gnu.build-id not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .dynsym not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .gnu.version not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .gnu.version_r not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .gnu.hash not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .dynstr not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .rela.dyn not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .relr.dyn not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .rela.plt not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .rodata not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .eh_frame_hdr not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .eh_frame not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .plt not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .data.rel.ro not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .fini_array not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .dynamic not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .got not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .got.plt not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .data not found in .gnu_debugdata for /system/lib64/libnetd_client.so
warning: section .bss not found in .gnu_debugdata for /system/lib64/libnetd_client.so

Program received signal SIGILL, Illegal instruction.
0x00000055555ba5f0 in std::__ndk1::enable_if<__is_cpp17_forward_iterator<std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >*>::value&&is_constructible<std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >, std::__ndk1::iterator_traits<std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >*>::reference>::value, void>::type std::__ndk1::vector<std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >, std::__ndk1::allocator<std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > > >::assign<std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >*>(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >*, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >*) ()
#0  0x00000055555ba5f0 in std::__ndk1::enable_if<__is_cpp17_forward_iterator<std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >*>::value&&is_constructible<std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >, std::__ndk1::iterator_traits<std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >*>::reference>::value, void>::type std::__ndk1::vector<std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >, std::__ndk1::allocator<std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > > >::assign<std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >*>(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >*, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >*) ()
#1  0x00000055555b9d08 in gpt_params::operator=(gpt_params const&) ()
#2  0x00000055555b25a8 in llama_server_context::
loadModel(gpt_params const&)
#3  0x00000055555a5724 in main ()

staviq commented 11 months ago

Thread 1 "main" received signal SIGINT, Interrupt.

SIGINT typically comes from the system itself, and the only explaination that comes to my mind is that you ran out of memory in this case

Program received signal SIGILL, Illegal instruction.

This one looks "better", because the Makefile trick should help with it, check my comment earlier in this thread, edit Makefile and recompile the same way you did in that comment where you got that SIGILL.

theoctopusride commented 11 months ago

Thread 1 "main" received signal SIGINT, Interrupt.

SIGINT typically comes from the system itself, and the only explaination that comes to my mind is that you ran out of memory in this case

I interrupted it myself. It was allowing me to input text. See the "hi" next to the carrot (>) where it allowed me to enter text and hit enter. And there were responses provided before and after the carrot appeared. This phone has 12 GB of RAM. That's why I'm trying to make it work on this particular phone.

Program received signal SIGILL, Illegal instruction.

This one looks "better", because the Makefile trick should help with it, check my comment earlier in this thread, edit Makefile and recompile the same way you did in that comment where you got that SIGILL.

Ok I will try this later today.

theoctopusride commented 11 months ago

@ElvisClaros Hmm... I cannot reproduce, it works fine for me. The only thing that comes to my mind is that CPU capabilities are incorrectly detected by the compiler in your case.

Can you try editing Makefile and commenting out or removing those lines, and compiling again ?

https://github.com/ggerganov/llama.cpp/blob/11dc1091f64b24ca6d643acc6d0051117ba60161/Makefile#L310-L311

yes this method works and is repeatable with make -C llama.cpp -j4 and llama.cpp/server -m /sdcard/Download/llama-2-7b-chat.Q3_K_S.gguf on the Honor Magic 5. Thank you!

staviq commented 11 months ago

@theoctopusride

Could you do one more test for me ?

What's the output from llc --version ? Specifically the value for Host CPU

theoctopusride commented 11 months ago

@theoctopusride

Could you do one more test for me ?

What's the output from llc --version ? Specifically the value for Host CPU

LLVM (http://llvm.org/):
  LLVM version 17.0.2
  Optimized build.
  Default target: aarch64-unknown-linux-android24
  Host CPU: cortex-x3

  Registered Targets:
    aarch64     - AArch64 (little endian)
    aarch64_32  - AArch64 (little endian ILP32)
    aarch64_be  - AArch64 (big endian)
    amdgcn      - AMD GCN GPUs
    arc         - ARC
    arm         - ARM
    arm64       - ARM64 (little endian)
    arm64_32    - ARM64 (little endian ILP32)
    armeb       - ARM (big endian)
    avr         - Atmel AVR Microcontroller
    bpf         - BPF (host endian)
    bpfeb       - BPF (big endian)
    bpfel       - BPF (little endian)
    csky        - C-SKY
    hexagon     - Hexagon
    lanai       - Lanai
    loongarch32 - 32-bit LoongArch
    loongarch64 - 64-bit LoongArch
    m68k        - Motorola 68000 family
    mips        - MIPS (32-bit big endian)
    mips64      - MIPS (64-bit big endian)
    mips64el    - MIPS (64-bit little endian)
    mipsel      - MIPS (32-bit little endian)
    msp430      - MSP430 [experimental]
    nvptx       - NVIDIA PTX 32-bit
    nvptx64     - NVIDIA PTX 64-bit
    ppc32       - PowerPC 32
    ppc32le     - PowerPC 32 LE
    ppc64       - PowerPC 64
    ppc64le     - PowerPC 64 LE
    r600        - AMD GPUs HD2XXX-HD6XXX
    riscv32     - 32-bit RISC-V
    riscv64     - 64-bit RISC-V
    sparc       - Sparc
    sparcel     - Sparc LE
    sparcv9     - Sparc V9
    systemz     - SystemZ
    thumb       - Thumb
    thumbeb     - Thumb (big endian)
    ve          - VE
    wasm32      - WebAssembly 32-bit
    wasm64      - WebAssembly 64-bit
    x86         - 32-bit X86: Pentium-Pro and above
    x86-64      - 64-bit X86: EM64T and AMD64
    xcore       - XCore

theoctopusride commented 11 months ago

@ElvisClaros Hmm... I cannot reproduce, it works fine for me. The only thing that comes to my mind is that CPU capabilities are incorrectly detected by the compiler in your case. Can you try editing Makefile and commenting out or removing those lines, and compiling again ? https://github.com/ggerganov/llama.cpp/blob/11dc1091f64b24ca6d643acc6d0051117ba60161/Makefile#L310-L311

yes this method works and is repeatable with make -C llama.cpp -j4 and llama.cpp/server -m /sdcard/Download/llama-2-7b-chat.Q3_K_S.gguf on the Honor Magic 5. Thank you!

issue closed