Open liechtym opened 4 months ago
I have a use case where I need to modify input embeddings before they
from ctransformers import AutoModelForCausalLM llm = AutoModelForCausalLM.from_pretrained("TheBloke/Llama-2-7b-Chat-GGUF", model_file="llama-2-7b-chat.Q4_K_M.gguf", model_type="llama", gpu_layers=50) embeddings = llm.embed('Some text') # Code to modify embeddings modified_embeddings = some_function(embeddings) res = llm.generate(inputs_embeds=modified_embeddings)
Is there any way to do this?
I have a use case where I need to modify input embeddings before they
Is there any way to do this?