search

dranger003 / llama.cpp-dotnet

Minimal C# bindings for llama.cpp + .NET core library with API host/client.
MIT License
66 stars 7 forks source link

trapping the meta parameters #25

Open HarvieKrumpet opened 3 weeks ago

HarvieKrumpet commented 3 weeks ago

Have a need to see some of these values dumped by llama.cpp on a model. I attempted to redirect the stdout to a custom routine. from the llama dll but it appears you didnt implement the llama_set_log function in native.

Never found a formal way to do this through the usual api calls either.

below are just a few. how can I do it through your api?


llm_load_print_meta: format           = GGUF V3 (latest)
llm_load_print_meta: arch             = gemma2
llm_load_print_meta: vocab type       = SPM
llm_load_print_meta: n_vocab          = 256000
llm_load_print_meta: n_merges         = 0
llm_load_print_meta: vocab_only       = 0
llm_load_print_meta: n_ctx_train      = 8192
llm_load_print_meta: n_embd           = 3584
llm_load_print_meta: n_layer          = 42
llm_load_print_meta: n_head           = 16
llm_load_print_meta: n_head_kv        = 8
llm_load_print_meta: n_rot            = 256
llm_load_print_meta: n_swa            = 4096
llm_load_print_meta: n_embd_head_k    = 256
llm_load_print_meta: n_embd_head_v    = 256
llm_load_print_meta: n_gqa            = 2
dranger003 commented 3 weeks ago

Take a look at the latest commit, I added a sample to dump the meta data.

HarvieKrumpet commented 3 weeks ago

works! thanks, already incorporated it into the my main loader, but missing some values that I wanted...

llm_load_print_meta: n_vocab = 256000 llm_load_print_meta: n_ctx_train = 8192 llm_load_print_meta: n_embd = 3584 llm_load_print_meta: n_layer = 42

dranger003 commented 3 weeks ago

You need to use the separate functions for those. llama_n_vocab llama_n_ctx_train llama_n_embd llama_n_layer