Error: ‘ggml_cuda_load_data’ was not declared in this scope;

[cdeisler]

Pulled latest with updated llama.cpp in the talk-llama example.

Build is failing on:
https://github.com/ggerganov/whisper.cpp/blob/master/examples/talk-llama/llama.cpp#L1116

WHISPER_CUBLAS=1 make talk-llama

Did I miss something on the latest? I can't find this signature in ggml-cuda.h.

examples/talk-llama/llama.cpp: In function ‘void llama_model_load_internal(const string&, llama_context&, int, int, ggml_type, bool, bool, bool, llama_progress_callback, void*)’:
examples/talk-llama/llama.cpp:1116:13: error: ‘ggml_cuda_load_data’ was not declared in this scope; did you mean ‘ggml_cuda_free_data’?
 1116 |             ggml_cuda_load_data(fname.c_str(), lt.ggml_tensor, lt.shards.at(0).file_off);
      |             ^~~~~~~~~~~~~~~~~~~
      |             ggml_cuda_free_data
make: *** [Makefile:290: talk-llama] Error 1

[Jiminator]

I got this error too. Lmk if you figure out how to fix this issue. What's interesting is that the build worked a few days ago on a different device for me, so it must be due to a recent commit.

[chrononaut79]

I've got the same error here.

[dereklll]

any solution?

[chrononaut79]

It's been awhile, but I think all I did was a clean install of CUDA. I also set my cuda arch in the makefile from 'native' to 'sm_87' as that's what my setup is, and Ubuntu installs a relatively old CUDA version that doesn't like the native argument. I'm fairly certain that's all I did. Whisper is the greatest thanks for all the amazing work you're doing.

On Fri, Aug 25, 2023 at 2:02 AM dereklll @.***> wrote:

any solution?

— Reply to this email directly, view it on GitHub https://github.com/ggerganov/whisper.cpp/issues/1055#issuecomment-1692811584, or unsubscribe https://github.com/notifications/unsubscribe-auth/A7JJSYONKD4QQ6CDB4HGB2DXXA5XLANCNFSM6AAAAAAZU6AXAI . You are receiving this because you commented.Message ID: @.***>

[smldis]

No, this update 5feb0df broke this example. Since then, ggml_cuda_load_data is not available anymore in ggml-cuda.cu . Anybody knows hot to fix this example?

(maybe another update of llama.cpp in the example dir?)

[dereklll]

It's been awhile, but I think all I did was a clean install of CUDA. I also set my cuda arch in the makefile from 'native' to 'sm_87' as that's what my setup is, and Ubuntu installs a relatively old CUDA version that doesn't like the native argument. I'm fairly certain that's all I did. Whisper is the greatest thanks for all the amazing work you're doing. … On Fri, Aug 25, 2023 at 2:02 AM dereklll @.> wrote: any solution? — Reply to this email directly, view it on GitHub <#1055 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/A7JJSYONKD4QQ6CDB4HGB2DXXA5XLANCNFSM6AAAAAAZU6AXAI . You are receiving this because you commented.Message ID: @.>

ifdef WHISPER_CUBLAS ifeq ($(shell expr $(NVCC_VERSION) >= 11.6), 1) CUDA_ARCH_FLAG=native else -CUDA_ARCH_FLAG=all +CUDA_ARCH_FLAG=sm_80 endif

-CFLAGS += -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I$(CUDA_PATH)/targets/$(UNAME_M)-linux/include -CXXFLAGS += -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I$(CUDA_PATH)/targets/$(UNAME_M)-linux/include -LDFLAGS += -lcublas -lculibos -lcudart -lcublasLt -lpthread -ldl -lrt -L/usr/local/cuda/lib64 -L/opt/cuda/lib64 -L$(CUDA_PATH)/targets/$(UNAME_M)-linux/lib +#CFLAGS += -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I$(CUDA_PATH)/targets/$(UNAME_M)-linux/include +#CXXFLAGS += -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I$(CUDA_PATH)/targets/$(UNAME_M)-linux/include +#LDFLAGS += -lcublas -lculibos -lcudart -lcublasLt -lpthread -ldl -lrt -L/usr/local/cuda/lib64 -L/opt/cuda/lib64 -L$(CUDA_PATH)/targets/$(UNAME_M)-linux/lib +CFLAGS += -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I$(CUDA_PATH)/targets/$(UNAME_M)-linux/include -I$(CUDA_PATH)/include +CXXFLAGS += -DGGML_USE_CUBLAS -I/usr/local/cuda/include -I/opt/cuda/include -I$(CUDA_PATH)/targets/$(UNAME_M)-linux/include -I$(CUDA_PATH)/include +LDFLAGS += -lcublas -lculibos -lcudart -lcublasLt -lpthread -ldl -lrt -L/usr/local/cuda/lib64 -L/opt/cuda/lib64 -L$(CUDA_PATH)/targets/$(UNAME_M)-linux/lib -L$(CUDA_PATH)/lib -L$(CUDART_PATH)/lib + WHISPER_OBJ += ggml-cuda.o NVCC = nvcc NVCCFLAGS = --forward-unknown-to-host-compiler -arch=$(CUDA_ARCH_FLAG) @@ -211,8 +215,8 @@ ifdef WHISPER_GPROF endif

ifneq ($(filter aarch64%,$(UNAME_M)),) -CFLAGS += -mcpu=native -CXXFLAGS += -mcpu=native +#CFLAGS += -mcpu=native +#CXXFLAGS += -mcpu=native endif