-
Following error occurred at request time:
```
CUDA error: an illegal memory access was encountered
```
Repro context:
- Mixtral-8x7b
- Adapter (rank 8)
- Long prompt
- Sharded (2+ GPUs)
…
-
### System Info
- GPU (Nvidia GeForce RTX 4070 Ti)
- CPU 13th Gen Intel(R) Core(TM) i5-13600KF
- 32 GB RAM
- 1TB SSD
- OS Windows 11
Package versions:
- TensorRT version 9.2.0.post12.dev5
…
-
### Your current environment
```text
2024-06-19 17:30:02,514 - [Collecting environment information...
PyTorch version: 2.3.0
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to b…
-
## Description
I am using the jupyter ai extension with a custom model provider as per steps in https://jupyter-ai.readthedocs.io/en/latest/users/index.html#custom-model-providers
However th…
-
Refer to the API docs in https://github.com/ollama/ollama/blob/main/docs/api.md , currently the response data format is not compatible with OpenAI API.
It is import to be compatible with OpenAI…
-
Is it possible to provide an API the mimics the functionality of the OPENAI API?
-
## Issue
* When dealing with fresh docker containers and assuming say the use of no volumes, there is no Model within the Ollama container.
* Rather than depending on the `MODEL` environment variabl…
-
### What is the issue?
ollama run codellama:34b
error occurred:
pulling manifest
pulling f36b668ebcd3... 100% ▕█████████████████████████████████████████████████████████████████████████████████…
-
**What's Happening**
When attempting to download the 70B-chat model using download.sh, the model itself returns a 403 forbidden code.
**Traceback**
*Note the the policy has been removed to mainta…
-
Hi,
At first thank you very much about this great plugin. It is great to have the possibility to test LLMs out of the IDE. This is great stuff.
Unfortunately I have also an issue. At first I had…