search

LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
4.66k stars 334 forks source link

Fails to Build on Apple Silicon, Deprecated Dependancy on "stat64"? #201

Closed gafei closed 1 year ago

gafei commented 1 year ago

Latest commit, as of 5/28, 28f1196f65eb27a8f231fb83b38c91fb42a1a11a fails to build on a Apple Silicon Mac. Seems to be a deprecated dependency on stat64, that didn't exist in prior versions.

Reverting to commit, 8b8f2f4 does build correctly on an Apple Silicon Mac.

Configuration: Macbook Air M2, 16GB, 1TB MacOS Ventura 13.3

python3 --version Python 3.11.3

make --version GNU Make 3.81 Copyright (C) 2006 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

This program built for i386-apple-darwin11.3.0

g++ --version Apple clang version 14.0.0 (clang-1400.0.29.202) Target: arm64-apple-darwin22.4.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

uname -a Darwin xxx.lan 22.4.0 Darwin Kernel Version 22.4.0: Mon Mar 6 21:01:02 PST 2023; root:xnu-8796.101.5~3/RELEASE_ARM64_T8112 arm64

"make" output is below:

I llama.cpp build info: I UNAME_S: Darwin I UNAME_P: arm I UNAME_M: arm64 I CFLAGS: -I. -I./include -I./include/CL -I./otherarch -I./otherarch/tools -Ofast -DNDEBUG -std=c11 -fPIC -pthread -s -pthread -DGGML_USE_ACCELERATE I CXXFLAGS: -I. -I./examples -I./include -I./include/CL -I./otherarch -I./otherarch/tools -O3 -DNDEBUG -std=c++11 -fPIC -pthread -s -Wno-multichar -Wno-write-strings -pthread I LDFLAGS: -framework Accelerate I CC: Apple clang version 14.0.0 (clang-1400.0.29.202) I CXX: Apple clang version 14.0.0 (clang-1400.0.29.202)

cc -I. -I./include -I./include/CL -I./otherarch -I./otherarch/tools -Ofast -DNDEBUG -std=c11 -fPIC -pthread -s -pthread -DGGML_USE_ACCELERATE -c ggml.c -o ggml.o clang: warning: argument unused during compilation: '-s' [-Wunused-command-line-argument] cc -I. -I./include -I./include/CL -I./otherarch -I./otherarch/tools -Ofast -DNDEBUG -std=c11 -fPIC -pthread -s -pthread -DGGML_USE_ACCELERATE -c otherarch/ggml_v2.c -o ggml_v2.o clang: warning: argument unused during compilation: '-s' [-Wunused-command-line-argument] cc -I. -I./include -I./include/CL -I./otherarch -I./otherarch/tools -Ofast -DNDEBUG -std=c11 -fPIC -pthread -s -pthread -DGGML_USE_ACCELERATE -c otherarch/ggml_v1.c -o ggml_v1.o clang: warning: argument unused during compilation: '-s' [-Wunused-command-line-argument] c++ -I. -I./examples -I./include -I./include/CL -I./otherarch -I./otherarch/tools -O3 -DNDEBUG -std=c++11 -fPIC -pthread -s -Wno-multichar -Wno-write-strings -pthread -c expose.cpp -o expose.o clang: warning: argument unused during compilation: '-s' [-Wunused-command-line-argument] In file included from expose.cpp:20: ./expose.h:4:8: warning: struct 'load_model_inputs' does not declare any constructor to initialize its non-modifiable members struct load_model_inputs ^ ./expose.h:6:15: note: const member 'threads' will never be initialized const int threads; ^ ./expose.h:7:15: note: const member 'blasthreads' will never be initialized const int blasthreads; ^ ./expose.h:8:15: note: const member 'max_context_length' will never be initialized const int max_context_length; ^ ./expose.h:9:15: note: const member 'batch_size' will never be initialized const int batch_size; ^ ./expose.h:10:16: note: const member 'f16_kv' will never be initialized const bool f16_kv; ^ ./expose.h:14:16: note: const member 'use_mmap' will never be initialized const bool use_mmap; ^ ./expose.h:15:16: note: const member 'use_mlock' will never be initialized const bool use_mlock; ^ ./expose.h:16:16: note: const member 'use_smartcontext' will never be initialized const bool use_smartcontext; ^ ./expose.h:17:16: note: const member 'unban_tokens' will never be initialized const bool unban_tokens; ^ ./expose.h:20:16: note: const member 'debugmode' will never be initialized const bool debugmode; ^ ./expose.h:24:8: warning: struct 'generation_inputs' does not declare any constructor to initialize its non-modifiable members struct generation_inputs ^ ./expose.h:26:15: note: const member 'seed' will never be initialized const int seed; ^ ./expose.h:28:15: note: const member 'max_context_length' will never be initialized const int max_context_length; ^ ./expose.h:29:15: note: const member 'max_length' will never be initialized const int max_length; ^ ./expose.h:30:17: note: const member 'temperature' will never be initialized const float temperature; ^ ./expose.h:31:15: note: const member 'top_k' will never be initialized const int top_k; ^ ./expose.h:33:17: note: const member 'top_p' will never be initialized const float top_p; ^ ./expose.h:34:17: note: const member 'typical_p' will never be initialized const float typical_p; ^ ./expose.h:35:17: note: const member 'tfs' will never be initialized const float tfs; ^ ./expose.h:36:17: note: const member 'rep_pen' will never be initialized const float rep_pen; ^ ./expose.h:37:15: note: const member 'rep_pen_range' will never be initialized const int rep_pen_range; ^ ./expose.h:39:17: note: const member 'mirostat_eta' will never be initialized const float mirostat_eta; ^ ./expose.h:40:17: note: const member 'mirostat_tau' will never be initialized const float mirostat_tau; ^ expose.cpp:193:24: warning: 'generate' has C-linkage specified, but returns user-defined type 'generation_outputs' which is incompatible with C [-Wreturn-type-c-linkage] generation_outputs generate(const generation_inputs inputs, generation_outputs &output) ^ 3 warnings generated. c++ -I. -I./examples -I./include -I./include/CL -I./otherarch -I./otherarch/tools -O3 -DNDEBUG -std=c++11 -fPIC -pthread -s -Wno-multichar -Wno-write-strings -pthread -c examples/common.cpp -o common.o clang: warning: argument unused during compilation: '-s' [-Wunused-command-line-argument] c++ -I. -I./examples -I./include -I./include/CL -I./otherarch -I./otherarch/tools -O3 -DNDEBUG -std=c++11 -fPIC -pthread -s -Wno-multichar -Wno-write-strings -pthread -c gpttype_adapter.cpp -o gpttype_adapter.o clang: warning: argument unused during compilation: '-s' [-Wunused-command-line-argument] In file included from gpttype_adapter.cpp:11: In file included from ./model_adapter.h:14: ./expose.h:4:8: warning: struct 'load_model_inputs' does not declare any constructor to initialize its non-modifiable members struct load_model_inputs ^ ./expose.h:6:15: note: const member 'threads' will never be initialized const int threads; ^ ./expose.h:7:15: note: const member 'blasthreads' will never be initialized const int blasthreads; ^ ./expose.h:8:15: note: const member 'max_context_length' will never be initialized const int max_context_length; ^ ./expose.h:9:15: note: const member 'batch_size' will never be initialized const int batch_size; ^ ./expose.h:10:16: note: const member 'f16_kv' will never be initialized const bool f16_kv; ^ ./expose.h:14:16: note: const member 'use_mmap' will never be initialized const bool use_mmap; ^ ./expose.h:15:16: note: const member 'use_mlock' will never be initialized const bool use_mlock; ^ ./expose.h:16:16: note: const member 'use_smartcontext' will never be initialized const bool use_smartcontext; ^ ./expose.h:17:16: note: const member 'unban_tokens' will never be initialized const bool unban_tokens; ^ ./expose.h:20:16: note: const member 'debugmode' will never be initialized const bool debugmode; ^ ./expose.h:24:8: warning: struct 'generation_inputs' does not declare any constructor to initialize its non-modifiable members struct generation_inputs ^ ./expose.h:26:15: note: const member 'seed' will never be initialized const int seed; ^ ./expose.h:28:15: note: const member 'max_context_length' will never be initialized const int max_context_length; ^ ./expose.h:29:15: note: const member 'max_length' will never be initialized const int max_length; ^ ./expose.h:30:17: note: const member 'temperature' will never be initialized const float temperature; ^ ./expose.h:31:15: note: const member 'top_k' will never be initialized const int top_k; ^ ./expose.h:33:17: note: const member 'top_p' will never be initialized const float top_p; ^ ./expose.h:34:17: note: const member 'typical_p' will never be initialized const float typical_p; ^ ./expose.h:35:17: note: const member 'tfs' will never be initialized const float tfs; ^ ./expose.h:36:17: note: const member 'rep_pen' will never be initialized const float rep_pen; ^ ./expose.h:37:15: note: const member 'rep_pen_range' will never be initialized const int rep_pen_range; ^ ./expose.h:39:17: note: const member 'mirostat_eta' will never be initialized const float mirostat_eta; ^ ./expose.h:40:17: note: const member 'mirostat_tau' will never be initialized const float mirostat_tau; ^ In file included from gpttype_adapter.cpp:16: In file included from ./otherarch/llama_v2.cpp:8: ./otherarch/llama_v2-util.h:55:16: warning: attribute 'format' is ignored, place it after "struct" to apply attribute to type declaration [-Wignored-attributes] attribute((format(printf, 1, 2))) ^ In file included from gpttype_adapter.cpp:16: In file included from ./otherarch/llama_v2.cpp:9: ./otherarch/llama_v2.h:171:33: warning: 'legacy_llama_v2_tokenize' has C-linkage specified, but returns user-defined type 'std::vector' (aka 'vector') which is incompatible with C [-Wreturn-type-c-linkage] std::vector legacy_llama_v2_tokenize(struct llama_v2_context * ctx, const std::string & text, bool add_bos); ^ In file included from gpttype_adapter.cpp:20: ./otherarch/gptj_v2.cpp:298:52: warning: format specifies type 'int' but the argument has type 'int64_t' (aka 'long long') [-Wformat] func, name.data(), tensor->ne[0], tensor->ne[1], ne[0], ne[1]); ^~~~~ ./otherarch/gptj_v2.cpp:298:67: warning: format specifies type 'int' but the argument has type 'int64_t' (aka 'long long') [-Wformat] func, name.data(), tensor->ne[0], tensor->ne[1], ne[0], ne[1]); ^~~~~ In file included from gpttype_adapter.cpp:21: ./otherarch/gptj_v3.cpp:298:52: warning: format specifies type 'int' but the argument has type 'int64_t' (aka 'long long') [-Wformat] func, name.data(), tensor->ne[0], tensor->ne[1], ne[0], ne[1]); ^~~~~ ./otherarch/gptj_v3.cpp:298:67: warning: format specifies type 'int' but the argument has type 'int64_t' (aka 'long long') [-Wformat] func, name.data(), tensor->ne[0], tensor->ne[1], ne[0], ne[1]); ^~~~~ In file included from gpttype_adapter.cpp:23: ./otherarch/gpt2_v2.cpp:291:78: warning: format specifies type 'long long' but the argument has type 'int32_t' (aka 'int') [-Wformat] func, name.data(), tensor->ne[0], tensor->ne[1], ne[0], ne[1]); ^~~~~ ./otherarch/gpt2_v2.cpp:291:85: warning: format specifies type 'long long' but the argument has type 'int32_t' (aka 'int') [-Wformat] func, name.data(), tensor->ne[0], tensor->ne[1], ne[0], ne[1]); ^~~~~ In file included from gpttype_adapter.cpp:24: ./otherarch/gpt2_v3.cpp:294:78: warning: format specifies type 'long long' but the argument has type 'int32_t' (aka 'int') [-Wformat] func, name.data(), tensor->ne[0], tensor->ne[1], ne[0], ne[1]); ^~~~~ ./otherarch/gpt2_v3.cpp:294:85: warning: format specifies type 'long long' but the argument has type 'int32_t' (aka 'int') [-Wformat] func, name.data(), tensor->ne[0], tensor->ne[1], ne[0], ne[1]); ^~~~~ In file included from gpttype_adapter.cpp:25: ./otherarch/rwkv_v2.cpp:140:5: warning: 'sprintf' is deprecated: This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations] sprintf(full_key, "blocks.%d.%s", block_index, key); ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:188:1: note: 'sprintf' has been explicitly marked deprecated here __deprecated_msg("This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead.") ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:215:48: note: expanded from macro '__deprecated_msg'

define deprecated_msg(_msg) attribute((deprecated__(_msg)))

                                                  ^

In file included from gpttype_adapter.cpp:26: ./otherarch/rwkv_v3.cpp:214:5: warning: 'sprintf' is deprecated: This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations] sprintf(full_key, "blocks.%d.%s", block_index, key.c_str()); ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:188:1: note: 'sprintf' has been explicitly marked deprecated here __deprecated_msg("This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead.") ^ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:215:48: note: expanded from macro '__deprecated_msg'

define deprecated_msg(_msg) attribute((deprecated__(_msg)))

                                                  ^

In file included from gpttype_adapter.cpp:26: ./otherarch/rwkv_v3.cpp:480:19: error: variable has incomplete type 'struct stat64' struct stat64 file_stat; ^ ./otherarch/rwkv_v3.cpp:480:12: note: forward declaration of 'stat64' struct stat64 file_stat; ^ gpttype_adapter.cpp:418:75: warning: format specifies type 'unsigned int' but the argument has type 'unsigned long' [-Wformat] printf("\nRWKV old Init: State Buffer:%u, Logit Buffer:%u\n", statebufsiz, logitbufsiz); ~~ ^~~ %lu gpttype_adapter.cpp:418:88: warning: format specifies type 'unsigned int' but the argument has type 'unsigned long' [-Wformat] printf("\nRWKV old Init: State Buffer:%u, Logit Buffer:%u\n", statebufsiz, logitbufsiz); ~~ ^~~ %lu gpttype_adapter.cpp:446:71: warning: format specifies type 'unsigned int' but the argument has type 'unsigned long' [-Wformat] printf("\nRWKV Init: State Buffer:%u, Logit Buffer:%u\n", statebufsiz, logitbufsiz); ~~ ^~~ %lu gpttype_adapter.cpp:446:84: warning: format specifies type 'unsigned int' but the argument has type 'unsigned long' [-Wformat] printf("\nRWKV Init: State Buffer:%u, Logit Buffer:%u\n", statebufsiz, logitbufsiz); ~~ ^~~ %lu gpttype_adapter.cpp:932:16: warning: format string is not a string literal (potentially insecure) [-Wformat-security] printf(outstr.c_str()); ^~~~~~ gpttype_adapter.cpp:932:16: note: treat the string as an argument to avoid this printf(outstr.c_str()); ^ "%s", gpttype_adapter.cpp:945:107: warning: format specifies type 'int' but the argument has type 'std::vector::size_type' (aka 'unsigned long') [-Wformat] printf("\rProcessing Prompt%s (%d / %d tokens)", (blasmode ? " [BLAS]" : ""), input_consumed, embd_inp.size()); ~~ ^~~~~~~ %lu 20 warnings and 1 error generated. make: *** [gpttype_adapter.o] Error 1

LostRuins commented 1 year ago

Thanks for bringing this to my attention. I had just integrated this from upstream rwkv.cpp and have not tested it on apple devices. I will try to fix it asap.

Pinging @saharNooby too.

LoganDark commented 1 year ago

Thanks for bringing this to my attention. I had just integrated this from upstream rwkv.cpp and have not tested it on apple devices. I will try to fix it asap.

Pinging @saharNooby too.

I found this issue a few days ago and included a fix in my most recent PR https://github.com/saharNooby/rwkv.cpp/pull/74 but it is not merged yet, I have a couple checks to make before I mark it as ready for review

LostRuins commented 1 year ago

Hi @gafei can you check if my latest commit fixes this? thanks.

LoganDark commented 1 year ago

Hi @gafei can you check if my latest commit fixes this? thanks.

looks like an ok workaround for me. Once my PR is merged this should no longer be an issue (it should work on all platforms).

LostRuins commented 1 year ago

Yes I saw your PR. Eventually if it's merged into rwkv.cpp, then it will be propagated here the next time I pull upstream. This is just a quick fix to allow mac users to compile.

gafei commented 1 year ago

Hi @LostRuins , your latest commit from https://github.com/LostRuins/koboldcpp/commit/ea336bfa332301c9f520e08fa7a538cc60dc0c96 fixes it! Complies and runs correctly. Thanks!