-
### Checked other resources
- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a…
-
**Setup**
Machine: AWS Sagemaker ml.p4d.24xlarge
Model: https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1
Used Docker container image with the latest build of trt-llm (`0.8.0.dev2024011…
-
GPU. 2*V100
convert command
```shell
python build.py --model_dir /data/models/vicuna-13b-v1.5/vicuna-13b-v1.5/ \
--dtype float16 \
--use_gpt_attention_plugin flo…
-
Hello, I am currently working with the new LLaMA 3.1 models by Meta and they require the newer versions of transformers, optimum, and accelerate. I ran into compatibility issues with XTTS regarding th…
-
### System Info
- CPU architecture: x86_64
- GPU properties
- GPU name: NVIDIA A100
- GPU memory size: 40G
- Libraries
- TensorRT-LLM branch or tag: main
- TensorRT-LLM commit: 5d8ca2…
-
### What is the issue?
My setup is a 4x A100 80GB, 2TB ram, dual intel cpu. Ubuntu server 22.04.
On a previous version of ollama, the model llama3.1:405b was loaded in a reasonable amount of second…
-
`[02/02/2022-06:42:51] [TRT-LLM] [I] Serially build TensorRT engines.
[02/02/2022-06:42:52] [TRT] [I] [MemUsageChange] Init CUDA: CPU +267, GPU +0, now: CPU 390, GPU 16638 (MiB)
[02/02/2022-06:42:53…
-
The command `python -m graphrag.index --root ./ragtest` runs successfully when the cache and output directories already exist.
However, after deleting the cache and output directories and running…
-
Hey guys,
I appreciated reading your paper. However I just wanted to see your eval results for myself and running the script results in the follwoing error when I execute ```python evaluate_hotpot_q…
-
Hello, I am facing an error when initializing LLMRails.
Here is the error: `srun: error: task 0: Illegal instruction (core dumped)`.
I think the error is related to `asyncio`. After a little debugg…