-
NIM updates have totally changed the way we interact with NVCF.
Example code for correct interaction with NIMs:
```
from openai import OpenAI
client = OpenAI(
base_url = "https://integrate…
-
TogetherAI just [announced ](https://www.together.ai/blog/function-calling-json-mode) JSON mode and function calling for their models. It currently supports these models: Mixtral, Mistra, and CodeLlam…
-
I saw that you mock `llama.cpp` but I still have gpu resources, event I also have enough cpu & RAM.
Just want to figure out it's right scene to deploying.
-
### Check for existing issues
- [X] Completed
### Describe the feature
Cut and Dry: Forget using copilot or chatgpt with their subscriptions and privacy nightmares.
We have generative transforme…
-
### System Info
Running a TGI 2.0.3 docker on a 8 NVIDIA_L4 VM.
Command:
```bash
MODEL=codellama/CodeLlama-70b-Python-hf
docker run \
-m 320G \
--shm-size=40G \
-e NVIDIA_VISIBLE_DEVIC…
-
Hi, I'm running the code generation command and reporting an error.
The triton version is consistent with the warehouse version. When I tried to upgrade the version, I still reported the same error. …
-
### Before submitting your bug report
- [X] I believe this is a bug. I'll try to join the [Continue Discord](https://discord.gg/NWtdYexhMs) for questions
- [X] I'm not able to find an [open issue]…
-
### Before submitting your bug report
- [X] I believe this is a bug. I'll try to join the [Continue Discord](https://discord.gg/NWtdYexhMs) for questions
- [X] I'm not able to find an [open issue](ht…
-
### Describe the bug
(codellama) amardeep.yadav@fintricity.com@codellamamachine:~$ pip install "openllm[vllm]"
Requirement already satisfied: openllm[vllm] in ./miniconda3/envs/codellama/lib/python3…
-
I've installed the extension, edited the Settings to use OpenAI for the model, code completion, and (ada) embedding. Added my OpenAI key. No Wingman features work. They fail silently except for the ho…