Closed ngxson closed 1 week ago
wllama.setOptions is added to support turn embeddings on/off dynamically. This feature is introduced in https://github.com/ggerganov/llama.cpp/pull/7477
wllama.setOptions
This feature involves:
llama_set_embeddings
llama_set_causal_attn
wllama.setOptionsis added to support turn embeddings on/off dynamically. This feature is introduced in https://github.com/ggerganov/llama.cpp/pull/7477This feature involves:
llama_set_embeddings==> true to return embeddings (and skip logits)llama_set_causal_attn==> true to return logits