-
### System Info
```
OS:
Operating System: Ubuntu 20.04.6 LTS
Kernel: Linux 5.4.0-146-generic
Architecture: x86-64
CPU:
Architecture: x86_64
CPU op-mode(s):…
-
### Prerequisites
- [X] I am running the latest code. Mention the version if possible as well.
- [X] I carefully followed the [README.md](https://github.com/ggerganov/llama.cpp/blob/master/README.md)…
-
### Contact Details
github
### What happened?
I came here to report the issue / bug / my incompetence around the error of: `llama_model_load: error loading model: done_getting_tensors: wrong numbe…
-
### Your current environment
```text
The output of `python collect_env.py`
PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
…
-
### System Info
NVIDIA-SMI 535.154.05
Driver Version: 535.154.05
CUDA Version: 12.4
- GPU properties
- GPU name: NVIDIA L20
- GPU memory size: 46068MiB
- Libraries
- Te…
-
### Your current environment
The output of `python collect_env.py`
Collecting environment information...
PyTorch version: 2.4.0
Is debug build: False
CUDA used to build PyTorch: 12.4
ROCM …
-
### System Info
- Architecture: x86_64
- OS Ubuntu 22.04
- GPU: NVIDIA GeForce RTX 4090
- Gpu memory 2x24gb
- CPU max MHz: 5000.0000
- Driver Version: 535.183.01
- CUDA Version: 12.2
- Conta…
-
/kind feature
**Describe the solution you'd like**
This proposal outlines the need for an API to standardize the discovery, installation, and aggregation of LLM functions, agents or tools in Kub…
-
ran the following command to install TRT-LLM on windows
```sh
pip install tensorrt_llm==0.11.0 --extra-index-url https://pypi.nvidia.com
```
Getting the following error. Followed instructions fr…
-
### Your current environment
```text
PyTorch version: 2.3.1+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: Alibaba Group Enterprise Linux Serv…