-
I learned that llama-cpp has an option to specify a GBNF grammar format.
The ability to specify formats precisely in this way, rather than through prompts, is very appealing, especially since I’m u…
-
### Feature Description
The GBNF works as a charm. Considering the consumer end are mostly typed, wouldn't it be better to support the conversion from type to grammar?
### The Solution
A simp…
-
I've been trying to use the package. Trying the example shown here throws the said error:
https://github.com/Maximilian-Winter/llama-cpp-agent/blob/master/examples/02_Structured_Output/book_dataset…
-
# Prerequisites
On b1557
# Expected Behavior
The model should generate output as normal, as defined in the grammar file. This appears to only impact deepseek, as llama variants and yi run fi…
-
Currently the Sampler does not actually support sampling with grammar what's currently in place just seems like a placeholder and not actually used in sampling. I'm not sure if this fits directly in w…
-
Running the sample wav file with a grammar gives no change in the output compared to without the grammar. I'm purposefully not giving any prompt because I want to see how it works without the help of …
-
I am unsure of why such a thing occurs. I'm passing in the json.gbnf grammar to restrict the output. I have also set the following parameters -
- seed: 123
- temperature: 0.0
- n_predict: 512
-…
-
I'm running Llama 3 with two A40s and am finding the `llama-cpp-agent` has a high sample time. To use the book example I find my sample time for creating an object is an order of magnitude slower. (I'…
-
Are there any instructions for using local models rather than GPT-3 or 4? Is there a way to set the basepath to 127.0.0.1:11435 to use ollama or to 1234/v2 for LM Studio? Is there a configuration fi…
-
Hi! I see you're drowning in issues and PRs here. :)
Partly as a follow-up to #2623, I'd like to add support for arbitrary GBNF. I'm going to do this for myself regardless.
The question is: Shou…