-
We've always referred to netnet as an "AI-TA" (artificial intelligence teaching assistant)... but that was before the rise of LLMs... now AI has all sorts of connotations we need to contend with.
…
nbriz updated
1 month ago
-
when I convert chatglm2-6b with A10, there is error as below:
Traceback (most recent call last):
File "/code/tensorrt-llm/tensorrt-llm/TensorRT-LLM/examples/chatglm/build.py", line 895, in
r…
-
### Describe the bug
bolt.new-any-llm> pnpm run dev
> bolt@ dev I:\Ai_Lab2\bolt.new-any-llm
> remix vite:dev
➜ Local: http://localhost:5173/
➜ Network: use --host to expose
➜ pres…
-
### System Info
CPU x86_64
GPU L40s
TensorRT branch: main
commid id:b57221b764bc579cbb2490154916a871f620e2c4
CUDA:
| NVIDIA-SMI 535.154.05 Driver Version: 535.154.05 CUDA V…
-
Many LLMs are trained with bf16, if we want to use the hidden states of LLMs for retrieval, those vectors will be in bf16 dtype. It would be helpful to support bf16 in Faiss so that we can use LLMs as…
-
### Bug Description
llamaindex version=0.10.68
baseurl and bearertoke
### Version
0.10.68
### Steps to Reproduce
using llmrerank
### Relevant Logs/Tracbacks
_No response_
-
While this projects contains the SOTA in terms of embeddings and local LLMs, the answer extraction is too slow.
It might be possible to use an answer extraction system based on RNNs (like Mamba) tr…
DrDub updated
2 months ago
-
### Your current environment
Collecting environment information...
/home/miniconda3/envs/vllm/lib/python3.12/site-packages/torch/cuda/init.py:128: UserWarning: CUDA initialization: Unexpected error …
-
**Is your feature request related to a problem? Please describe:**
Congrats on the launch! Very cool stuff, but one immediate limitation I noticed is you don't have realtime info about packages. li…
-
### System Info
ubuntu 20.04
tensorrt 10.0.1
tensorrt-cu12 10.0.1
tensorrt-cu12-bindings 10.0.1
tensorrt-cu12-libs 10.0.1
tensorrt-llm 0.11.0.dev2024052100
nvidia L40s
### Who can help?
…