-
To apply grammar to chat completion, it looks like the llamafile server is expecting the argument `grammar`: https://github.com/Mozilla-Ocho/llamafile/blob/main/llama.cpp/server/server.cpp#L2551
```
…
-
I tried added a custom document and chat with it but it doesn't process. Am i missing something in my local setup?
Here is the snippet of what i tried.
```
(llama-app-backend-py3.11) ➜ backend g…
-
How can I use the newly release models from meta with Ollama?
https://huggingface.co/collections/facebook/llm-compiler-667c5b05557fe99a9edd25cb
Thanks.
-
### Model description
Hi all,
I'm attempting to convert [Llama-3](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) to ONNX format.
### Prerequisites
- [X] The model is supported in …
-
Hi @sfeucht congrats on this work!
Would you be interested in pushing your datasets to the hub? See here on how to do that: https://huggingface.co/docs/datasets/loading#csv. You can then simply do …
-
I have discovered that running the same model with the same parameters from llm (gguf branch) and llama.cpp results in a different behavior. llm seems to have not been reading EOS token and thus the …
-
I have a consistent error when creating a new model based on my local gguf file. This error only happens when creating a first model, i.e. if there are any already loaded models in ollama this does no…
-
### Issue you'd like to raise.
Hey!
I'm playing with "enabled API server" and if I send request via curl
Request:
```
curl --request POST \
--url http://localhost:4891/v1/completions \
--…
-
I followed the step-by-step instructions but upon running make run to check if it all worked, I get the following error message:
File "/home/ubuntu/.cache/pypoetry/virtualenvs/private-gpt-Wtvj2B-w-…
-
### Summary
# Motivation
The WasmEdge Runtime is poised to offer robust inference support for AI models and Large
Language Models (LLMs) such as llama3 and phi-3-mini. Recognizing the critical …