-
### System Info
### System Info
GPU: NVIDIA A100
Driver Version: 545.23.08
CUDA: 12.3
versions:
https://github.com/NVIDIA/TensorRT-LLM.git (5fa9436) (latest version)
https://github.com/trit…
-
### Reminder
- [X] I have read the README and searched the existing issues.
### Reproduction
Hi, I am quite new to llama factory framework, I am not able to find the config.yaml for longlora and st…
-
I have a question on paper results.
![image](https://github.com/Infini-AI-Lab/TriForce/assets/50622684/d69216c5-1b99-466e-b1e6-b1134b140abc)
Does Retrieval w/o Hierarchy test with normal speculati…
bxyb updated
5 months ago
-
### System Info
3090 server
### Who can help?
_No response_
### Information
- [ ] The official example scripts
- [ ] My own modified scripts
### Tasks
- [ ] An officially supported task in the …
-
### System Info
- NVIDIA A100 80G * 2
- Libraries
- TensorRT-LLM 0.11.0
- Driver Version: 525.105.17
- CUDA Version: 12.4
### Who can help?
@byshiue
### Information
- [X] The official exa…
-
### System Info
4x NVIDIA H100, TensorRT-LLM backend 0.9.0
### Who can help?
@Tracin
### Information
- [X] The official example scripts
- [ ] My own modified scripts
### Tasks
-…
-
### System Info
- CPU Architecture x86_64
- GPU: NVIDIA H100
- TensorRT-LLM v0.10.0
### Who can help?
_No response_
### Information
- [ ] The official example scripts
- [ ] My own modified scri…
-
### System Info
- x86_64
- GPU Mem: 640GB
- CPU Mem: 1.5TB
- 8 * NVIDIA H100
- TensorRT-LLM Version: `0.12.0.dev2024072301`
- TensorRT-LLM Commit `5fa9436e17c2f9aeace070f49aa645d2577f676b`
- T…
-
modified :
tensorrt_llm.models.chatglm.model.py
def use_lora(self, lora_config: LoraConfig):
trtllm_modules_to_hf_modules = {
"attn_qkv": "query_key_value",
"att…
-
I'm trying to run the TensorRT version of the docker container according to instructions, but am getting a segfault whenever I attempt to transcribe any audio. The same audio works with the Faster whi…