-
Hi, I noticed that the following script produces different results depending on the backend. On my machine, the output is:
```julia
cpu: [18.0; 18.0; 18.0; 18.0; 18.0; 18.0; 18.0; 18.0; 18.0; 18.0…
-
```
import os
from crewai import Agent, Task, Crew, Process
from langchain_community.llms import Ollama
from crewai_tools import BaseTool
# Initialize the OpenAI LLM with the specific model
…
-
### What is the issue?
Hi,
I've tried to pull and build the Ollama git like I have done in the past, but the latest version doesn't seem to work when installed.
If I try to run a model I get
o…
-
This is when you usually will lose any type of verbose response in inferencing
`torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 15.33 GiB. GPU 0 has a total capacity of 24.00 GiB of w…
-
### What happened?
Qwen2-72B-Instruct Q4_K_M generates output with random tokens (numbers, special symbols, random chunks of words from different languages, etc).
Has been tested on:
1) Tesla P…
-
`python llama.cpp/convert-hf-to-gguf.py --outtype f16 --outfile /content/Phi-3-small-128k-instruct.f16.gguf /content/Phi-3-small-128k-instruct`
```
INFO:hf-to-gguf:Loading model: Phi-3-small-128k-…
-
### Your current environment
```text
The output of `python collect_env.py`
```
### 🐛 Describe the bug
Description:
When loading FP8 quantized models with merged linear modules (e.g., Phi…
mgoin updated
3 months ago
-
We've been testing running assistants with ollama. This is of interest for some folks using agency-swarm with assistants but may be interesting to others as well (assistants-api with local models).
…
phact updated
5 months ago
-
### What is the issue?
Regardless of what's in the modelfile, it seems phi3 doesn't take in the SYSTEM prompt at all. I've looked around and can't find anyone else discussing this. Assuming this is a…
rb81 updated
3 months ago
-
### What is the issue?
I'm getting the following error when testing the new 128k versions of phi3-medium:
```sh
$ ollama run phi3:14b-medium-128k-instruct-q4_0
Error: llama runner process has te…