Closed hiqsociety closed 11 months ago
in another related issue, i used llama.cpp that has compiled successfully with cublas and replaced the llama.cpp folder and tried to cublas make libbinding.a with error below, how to resolve?
root@ubuntu:/usr/local/src/go-llama.cpp# BUILD_TYPE=cublas make libbinding.a
I llama.cpp build info:
I UNAME_S: Linux
I UNAME_P: x86_64
I UNAME_M: x86_64
I CFLAGS: -I./llama.cpp -I. -O3 -DNDEBUG -std=c11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -Wno-unused-function -pthread -march=native -mtune=native
I CXXFLAGS: -I./llama.cpp -I. -I./llama.cpp/common -I./common -O3 -DNDEBUG -std=c++11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -pthread
I CGO_LDFLAGS:
I LDFLAGS:
I BUILD_TYPE: cublas
I CMAKE_ARGS: -DLLAMA_CUBLAS=ON
I EXTRA_TARGETS: llama.cpp/ggml-cuda.o
I CC: cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
I CXX: g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
g++ -I./llama.cpp -I. -I./llama.cpp/common -I./common -O3 -DNDEBUG -std=c++11 -fPIC -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -pthread -I./llama.cpp -I./llama.cpp/common binding.cpp -o binding.o -c
binding.cpp: In function ‘int get_embeddings(void*, void*, float*)’:
binding.cpp:39:5: error: ‘llama_binding_state’ was not declared in this scope; did you mean ‘llama_beams_state’?
39 | llama_binding_state* state = (llama_binding_state*) state_pr;
| ^~~~~~~~~~~~~~~~~~~
| llama_beams_state
binding.cpp:39:26: error: ‘state’ was not declared in this scope; did you mean ‘_xstate’?
39 | llama_binding_state* state = (llama_binding_state*) state_pr;
| ^~~~~
| _xstate
binding.cpp:39:55: error: expected primary-expression before ‘)’ token
39 | llama_binding_state* state = (llama_binding_state*) state_pr;
| ^
binding.cpp:37:44: warning: unused parameter ‘state_pr’ [-Wunused-parameter]
37 | int get_embeddings(void* params_ptr, void* state_pr, float * res_embeddings) {
| ~~~~~~^~~~~~~~
binding.cpp: In function ‘int get_token_embeddings(void*, void*, int*, int, float*)’:
binding.cpp:78:5: error: ‘llama_binding_state’ was not declared in this scope; did you mean ‘llama_beams_state’?
78 | llama_binding_state* state = (llama_binding_state*) state_pr;
| ^~~~~~~~~~~~~~~~~~~
| llama_beams_state
binding.cpp:78:26: error: ‘state’ was not declared in this scope; did you mean ‘_xstate’?
78 | llama_binding_state* state = (llama_binding_state*) state_pr;
| ^~~~~
| _xstate
binding.cpp:78:55: error: expected primary-expression before ‘)’ token
78 | llama_binding_state* state = (llama_binding_state*) state_pr;
| ^
binding.cpp: In function ‘int eval(void*, void*, char*)’:
binding.cpp:94:5: error: ‘llama_binding_state’ was not declared in this scope; did you mean ‘llama_beams_state’?
94 | llama_binding_state* state = (llama_binding_state*) state_pr;
| ^~~~~~~~~~~~~~~~~~~
| llama_beams_state
binding.cpp:94:26: error: ‘state’ was not declared in this scope; did you mean ‘_xstate’?
94 | llama_binding_state* state = (llama_binding_state*) state_pr;
| ^~~~~
| _xstate
binding.cpp:94:55: error: expected primary-expression before ‘)’ token
94 | llama_binding_state* state = (llama_binding_state*) state_pr;
| ^
binding.cpp:92:33: warning: unused parameter ‘state_pr’ [-Wunused-parameter]
92 | int eval(void* params_ptr,void* state_pr,char *text) {
| ~~~~~~^~~~~~~~
binding.cpp: In function ‘int llama_predict(void*, void*, char*, bool)’:
binding.cpp:120:5: error: ‘llama_binding_state’ was not declared in this scope; did you mean ‘llama_beams_state’?
120 | llama_binding_state* state = (llama_binding_state*) state_pr;
| ^~~~~~~~~~~~~~~~~~~
| llama_beams_state
binding.cpp:120:26: error: ‘state’ was not declared in this scope; did you mean ‘_xstate’?
120 | llama_binding_state* state = (llama_binding_state*) state_pr;
| ^~~~~
| _xstate
binding.cpp:120:55: error: expected primary-expression before ‘)’ token
120 | llama_binding_state* state = (llama_binding_state*) state_pr;
| ^
binding.cpp:332:53: warning: format ‘%zu’ expects argument of type ‘size_t’, but argument 2 has type ‘int’ [-Wformat=]
332 | printf("<<input too long: skipped %zu token%s>>", skipped_tokens, skipped_tokens != 1 ? "s" : "");
| ~~^ ~~~~~~~~~~~~~~
| | |
| | int
| long unsigned int
| %u
binding.cpp:464:42: warning: cast from type ‘const char*’ to type ‘char*’ casts away qualifiers [-Wcast-qual]
464 | if (!tokenCallback(state_pr, (char*)token_str.c_str())) {
| ^~~~~~~~~~~~~~~~~~~~~~~~
binding.cpp:290:10: warning: unused variable ‘is_antiprompt’ [-Wunused-variable]
290 | bool is_antiprompt = false;
| ^~~~~~~~~~~~~
binding.cpp:291:10: warning: unused variable ‘input_echo’ [-Wunused-variable]
291 | bool input_echo = true;
| ^~~~~~~~~~
binding.cpp:531:1: warning: label ‘end’ defined but not used [-Wunused-label]
531 | end:
| ^~~
binding.cpp: In function ‘int speculative_sampling(void*, void*, void*, char*, bool)’:
binding.cpp:556:5: error: ‘llama_binding_state’ was not declared in this scope; did you mean ‘llama_beams_state’?
556 | llama_binding_state* target_model_state = (llama_binding_state*) target_model;
| ^~~~~~~~~~~~~~~~~~~
| llama_beams_state
binding.cpp:556:26: error: ‘target_model_state’ was not declared in this scope; did you mean ‘target_model’?
556 | llama_binding_state* target_model_state = (llama_binding_state*) target_model;
| ^~~~~~~~~~~~~~~~~~
| target_model
binding.cpp:556:68: error: expected primary-expression before ‘)’ token
556 | llama_binding_state* target_model_state = (llama_binding_state*) target_model;
| ^
binding.cpp:557:26: error: ‘draft_model_state’ was not declared in this scope; did you mean ‘draft_model’?
557 | llama_binding_state* draft_model_state = (llama_binding_state*) draft_model;
| ^~~~~~~~~~~~~~~~~
| draft_model
binding.cpp:557:67: error: expected primary-expression before ‘)’ token
557 | llama_binding_state* draft_model_state = (llama_binding_state*) draft_model;
| ^
binding.cpp:654:45: warning: cast from type ‘const char*’ to type ‘char*’ casts away qualifiers [-Wcast-qual]
654 | if (!tokenCallback(draft_model, (char*)token_str.c_str())) {
| ^~~~~~~~~~~~~~~~~~~~~~~~
binding.cpp:563:19: warning: unused variable ‘model_tgt’ [-Wunused-variable]
563 | llama_model * model_tgt = target_model_state->model;
| ^~~~~~~~~
binding.cpp:564:19: warning: unused variable ‘model_dft’ [-Wunused-variable]
564 | llama_model * model_dft = draft_model_state->model;
| ^~~~~~~~~
binding.cpp:553:50: warning: unused parameter ‘target_model’ [-Wunused-parameter]
553 | int speculative_sampling(void* params_ptr, void* target_model, void* draft_model, char* result, bool debug) {
| ~~~~~~^~~~~~~~~~~~
binding.cpp: In function ‘void llama_binding_free_model(void*)’:
binding.cpp:795:5: error: ‘llama_binding_state’ was not declared in this scope; did you mean ‘llama_beams_state’?
795 | llama_binding_state* ctx = (llama_binding_state*) state_ptr;
| ^~~~~~~~~~~~~~~~~~~
| llama_beams_state
binding.cpp:795:26: error: ‘ctx’ was not declared in this scope
795 | llama_binding_state* ctx = (llama_binding_state*) state_ptr;
| ^~~
binding.cpp:795:53: error: expected primary-expression before ‘)’ token
795 | llama_binding_state* ctx = (llama_binding_state*) state_ptr;
| ^
binding.cpp:794:37: warning: unused parameter ‘state_ptr’ [-Wunused-parameter]
794 | void llama_binding_free_model(void *state_ptr) {
| ~~~~~~^~~~~~~~~
binding.cpp: In function ‘int llama_tokenize_string(void*, void*, int*)’:
binding.cpp:807:5: error: ‘llama_binding_state’ was not declared in this scope; did you mean ‘llama_beams_state’?
807 | llama_binding_state* state = (llama_binding_state*) state_pr;
| ^~~~~~~~~~~~~~~~~~~
| llama_beams_state
binding.cpp:807:26: error: ‘state’ was not declared in this scope; did you mean ‘_xstate’?
807 | llama_binding_state* state = (llama_binding_state*) state_pr;
| ^~~~~
| _xstate
binding.cpp:807:55: error: expected primary-expression before ‘)’ token
807 | llama_binding_state* state = (llama_binding_state*) state_pr;
| ^
binding.cpp:805:51: warning: unused parameter ‘state_pr’ [-Wunused-parameter]
805 | int llama_tokenize_string(void* params_ptr, void* state_pr, int* result) {
| ~~~~~~^~~~~~~~
binding.cpp: In function ‘void* load_model(const char*, int, int, bool, bool, bool, bool, bool, int, int, const char*, const char*, bool, float, float, bool, const char*, const char*, bool)’:
binding.cpp:946:11: error: ‘load_binding_model’ was not declared in this scope
946 | return load_binding_model(fname, n_ctx, n_seed, memory_f16, mlock, embeddings, mmap, low_vram, n_gpu_layers, n_batch, maingpu, tensorsplit, numa, rope_freq_base, rope_freq_scale, mul_mat_q, lora, lora_base, perplexity);
| ^~~~~~~~~~~~~~~~~~
make: *** [Makefile:207: binding.o] Error 1
You original output contains only warning; these happen with current head
but if works find despite them so you can just ignore them.
go-llama.cpp patches a few files inside the llama.cpp submodule, so you need to run that patch and compile llama.cpp (make -B prepare
from go-llama.cpp folder will force the patching if make doesn't detect that it should re-run it). I think you should just do what you did at first and ignore the warnings (also when they occur when you compile your go program).
@MathiasGS thx for the prompt reply,
i cant ignore them. i've done the make cublas libbinding.a thing and it works but when i try to run the example as shown, it throws out error. how to resolve?
# github.com/go-skynet/go-llama.cpp
binding.cpp: In function ‘int llama_predict(void*, void*, char*, bool)’:
binding.cpp:332:53: warning: format ‘%zu’ expects argument of type ‘size_t’, but argument 2 has type ‘int’ [-Wformat=]
332 | printf("<<input too long: skipped %zu token%s>>", skipped_tokens, skipped_tokens != 1 ? "s" : "");
| ~~^ ~~~~~~~~~~~~~~
| | |
| | int
| long unsigned int
| %u
binding.cpp: In function ‘void llama_binding_free_model(void*)’:
binding.cpp:797:5: warning: possible problem detected in invocation of ‘operator delete’ [-Wdelete-incomplete]
797 | delete ctx->model;
| ^~~~~~~~~~~~~~~~~
binding.cpp:797:17: warning: invalid use of incomplete type ‘struct llama_model’
797 | delete ctx->model;
| ~~~~~^~~~~
In file included from ../llama.cpp/common/common.h:5,
from binding.cpp:1:
../llama.cpp/llama.h:60:12: note: forward declaration of ‘struct llama_model’
60 | struct llama_model;
| ^~~~~~~~~~~
binding.cpp:797:5: note: neither the destructor nor the class-specific ‘operator delete’ will be called, even if they are declared when the class is defined
797 | delete ctx->model;
| ^~~~~~~~~~~~~~~~~
# github.com/go-skynet/go-llama.cpp/examples
/usr/local/go/pkg/tool/linux_amd64/link: running g++ failed: exit status 1
/usr/bin/ld: /tmp/go-link-1649869791/000002.o: in function `load_model':
/usr/local/src/go-llama.cpp/binding.cpp:946: undefined reference to `load_binding_model(char const*, int, int, bool, bool, bool, bool, bool, int, int, char const*, char const*, bool, float, float, bool, char const*, char const*, bool)'
/usr/bin/ld: /usr/local/src/go-llama.cpp//libbinding.a(ggml-cuda.o): in function `quantize_row_q8_1_cuda(float const*, void*, int, int, int, CUstream_st*)':
tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0xcb): undefined reference to `__cudaPushCallConfiguration'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x172): undefined reference to `__cudaPopCallConfiguration'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x1b1): undefined reference to `cudaLaunchKernel'
/usr/bin/ld: /usr/local/src/go-llama.cpp//libbinding.a(ggml-cuda.o): in function `ggml_cuda_cpy_tensor_2d(void*, ggml_tensor const*, long, long, long, long, CUstream_st*)':
tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x325): undefined reference to `cudaMemcpy2DAsync'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x391): undefined reference to `cudaGetDevice'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x407): undefined reference to `cudaMemcpy2DAsync'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x4a1): undefined reference to `cudaGetDevice'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x4a9): undefined reference to `cudaGetErrorString'
/usr/bin/ld: /usr/local/src/go-llama.cpp//libbinding.a(ggml-cuda.o): in function `ggml_cuda_pool_malloc(unsigned long, unsigned long*)':
tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x561): undefined reference to `cudaGetDevice'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x66f): undefined reference to `cudaMalloc'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x6d8): undefined reference to `cudaGetDevice'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x6e0): undefined reference to `cudaGetErrorString'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x73e): undefined reference to `cudaGetDevice'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x746): undefined reference to `cudaGetErrorString'
/usr/bin/ld: /usr/local/src/go-llama.cpp//libbinding.a(ggml-cuda.o): in function `ggml_cuda_pool_free(void*, unsigned long)':
tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x7d1): undefined reference to `cudaGetDevice'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x876): undefined reference to `cudaFree'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x88f): undefined reference to `cudaGetDevice'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x897): undefined reference to `cudaGetErrorString'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x8f5): undefined reference to `cudaGetDevice'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text+0x8fd): undefined reference to `cudaGetErrorString'
continuing...
tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st[_Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st]+0x2cc): undefined reference to `__cudaPushCallConfiguration'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st[_Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st]+0x3a9): undefined reference to `__cudaPushCallConfiguration'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st[_Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st]+0x47b): undefined reference to `__cudaPopCallConfiguration'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st[_Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st]+0x4d8): undefined reference to `cudaLaunchKernel'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st[_Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st]+0x569): undefined reference to `__cudaPushCallConfiguration'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st[_Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st]+0x63c): undefined reference to `__cudaPopCallConfiguration'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st[_Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st]+0x699): undefined reference to `cudaLaunchKernel'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st[_Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st]+0x766): undefined reference to `__cudaPopCallConfiguration'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st[_Z17ggml_cuda_op_ropePK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st]+0x7c3): undefined reference to `cudaLaunchKernel'
/usr/bin/ld: /usr/local/src/go-llama.cpp//libbinding.a(ggml-cuda.o): in function `ggml_cuda_op_mul_mat_cublas(ggml_tensor const*, ggml_tensor const*, ggml_tensor*, char const*, float const*, char const*, float*, long, long, long, long, CUstream_st* const&)':
tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z27ggml_cuda_op_mul_mat_cublasPK11ggml_tensorS1_PS_PKcPKfS4_PfllllRKP11CUstream_st[_Z27ggml_cuda_op_mul_mat_cublasPK11ggml_tensorS1_PS_PKcPKfS4_PfllllRKP11CUstream_st]+0xac): undefined reference to `cudaGetDevice'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z27ggml_cuda_op_mul_mat_cublasPK11ggml_tensorS1_PS_PKcPKfS4_PfllllRKP11CUstream_st[_Z27ggml_cuda_op_mul_mat_cublasPK11ggml_tensorS1_PS_PKcPKfS4_PfllllRKP11CUstream_st]+0xe3): undefined reference to `cublasSetStream_v2'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z27ggml_cuda_op_mul_mat_cublasPK11ggml_tensorS1_PS_PKcPKfS4_PfllllRKP11CUstream_st[_Z27ggml_cuda_op_mul_mat_cublasPK11ggml_tensorS1_PS_PKcPKfS4_PfllllRKP11CUstream_st]+0x134): undefined reference to `cublasSgemm_v2'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z27ggml_cuda_op_mul_mat_cublasPK11ggml_tensorS1_PS_PKcPKfS4_PfllllRKP11CUstream_st[_Z27ggml_cuda_op_mul_mat_cublasPK11ggml_tensorS1_PS_PKcPKfS4_PfllllRKP11CUstream_st]+0x314): undefined reference to `cudaGetDevice'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z27ggml_cuda_op_mul_mat_cublasPK11ggml_tensorS1_PS_PKcPKfS4_PfllllRKP11CUstream_st[_Z27ggml_cuda_op_mul_mat_cublasPK11ggml_tensorS1_PS_PKcPKfS4_PfllllRKP11CUstream_st]+0x373): undefined reference to `cudaGetDevice'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z27ggml_cuda_op_mul_mat_cublasPK11ggml_tensorS1_PS_PKcPKfS4_PfllllRKP11CUstream_st[_Z27ggml_cuda_op_mul_mat_cublasPK11ggml_tensorS1_PS_PKcPKfS4_PfllllRKP11CUstream_st]+0x392): undefined reference to `cudaGetDevice'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z27ggml_cuda_op_mul_mat_cublasPK11ggml_tensorS1_PS_PKcPKfS4_PfllllRKP11CUstream_st[_Z27ggml_cuda_op_mul_mat_cublasPK11ggml_tensorS1_PS_PKcPKfS4_PfllllRKP11CUstream_st]+0x39a): undefined reference to `cudaGetErrorString'
/usr/bin/ld: /usr/local/src/go-llama.cpp//libbinding.a(ggml-cuda.o): in function `ggml_cuda_op_alibi(ggml_tensor const*, ggml_tensor const*, ggml_tensor*, float const*, float const*, float*, CUstream_st* const&)':
tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z18ggml_cuda_op_alibiPK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st[_Z18ggml_cuda_op_alibiPK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st]+0x1e7): undefined reference to `__cudaPushCallConfiguration'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z18ggml_cuda_op_alibiPK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st[_Z18ggml_cuda_op_alibiPK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st]+0x2d5): undefined reference to `__cudaPopCallConfiguration'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text._Z18ggml_cuda_op_alibiPK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st[_Z18ggml_cuda_op_alibiPK11ggml_tensorS1_PS_PKfS4_PfRKP11CUstream_st]+0x314): undefined reference to `cudaLaunchKernel'
/usr/bin/ld: /usr/local/src/go-llama.cpp//libbinding.a(ggml-cuda.o): in function `__sti____cudaRegisterAll()':
tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text.startup+0xd): undefined reference to `__cudaRegisterFatBinary'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text.startup+0x41): undefined reference to `__cudaRegisterFunction'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text.startup+0x6f): undefined reference to `__cudaRegisterFunction'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text.startup+0x9d): undefined reference to `__cudaRegisterFunction'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text.startup+0xcb): undefined reference to `__cudaRegisterFunction'
/usr/bin/ld: tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text.startup+0xf9): undefined reference to `__cudaRegisterFunction'
/usr/bin/ld: /usr/local/src/go-llama.cpp//libbinding.a(ggml-cuda.o):tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text.startup+0x127): more undefined references to `__cudaRegisterFunction' follow
/usr/bin/ld: /usr/local/src/go-llama.cpp//libbinding.a(ggml-cuda.o): in function `__sti____cudaRegisterAll()':
tmpxft_00011567_00000000-6_ggml-cuda.compute_70.cudafe1.cpp:(.text.startup+0xd41): undefined reference to `__cudaRegisterFatBinaryEnd'
collect2: error: ld returned 1 exit status
It fails with undefined reference to
load_binding_model[...]
which is exactly the method being added by the patch (well, one of them). This leads me to believe again that the patch was not correctly applied. I think the "[I] replaced the llama.cpp folder" thing is biting you here and I would just delete the go-llama.cpp
folder and start from scratch with a new checkout and following the guide like you did originally. Seems to me that just ignoring the warnings would have left you with a working binary in your first try. Hope this gets you going!
@MathiasGS closing this as new issue: https://github.com/go-skynet/go-llama.cpp/issues/218
CGO_LDFLAGS="-lcublas -lcudart -L/usr/local/cuda/lib64/" LIBRARY_PATH=$PWD C_INCLUDE_PATH=$PWD go build ./examples/main.go