-
When will the batch_manager and executor of the cpp code be open sourced?
code:
TensorRT-LLM\cpp\tensorrt_llm\batch_manager\x86_64-linux-gnu
TensorRT-LLM\cpp\tensorrt_llm\executor\x86_64-linux-gnu
-
making this to discuss the issues in the new version of the prompt that was added to the repo.
prompt in discussion: https://github.com/NeoVertex1/SuperPrompt/edit/main/tm_prompt.md
this might h…
-
How to support the new model in cpp runtime ? Is there any reference document ? For example, the multimodal model [llava-one-vision](https://huggingface.co/lmms-lab/llava-onevision-qwen2-7b-ov)
Foll…
-
### Your current environment
```text
The output of `python collect_env.py`
```
CODE:
from langchain.llms import VLLM
import time
import uvicorn
app = FastAPI()
llm = VLLM(model="tiiua…
-
# Learn by Blogging - The Mental Model for Leveraging LLMs in Cloud
In this blog post, we are exploring the intersection of different sized LLMs and their optimal compute environments for deployment
…
-
I know it could take a while, or I did hear about a thing where Mozilla is working on a LLM to compete with openAI. I'm not sure if this is based on ChatGPT, but here's the link to the article. Making…
AYT04 updated
8 months ago
-
A game jam / hackathon around using LLMs in interesting ways, not to replace reading/writing, but to help us do it better. See ideas in https://github.com/DefenderOfBasic/works-in-progress/issues/7
…
-
I just run below code and find that the examples need to be evaluated by LLM are not equivalent to your papers.
`rule_based_source = ["E2E", "WIKIEVENTS", "CONLL2003",
"tex…
-
Paper: "our text generator, GPT-3 [6] (text-davinci-003, temp=0.5), generates the verbal cue".
![image](https://github.com/StephanAkkerman/FluentAI/assets/45365128/1a5f1acd-fbc9-4c8e-8bb6-cc641fa418a…
-
### Cortex version
0.5.1-rc2
### Describe the Bug
`cortex-beta run openhermes-2.5-7b-tensorrt-llm-linux-ada` fails with logs below.
### Steps to Reproduce
1. `cortex-beta run openhermes-2.5-7b-te…