-
version: TensorRT-LLM 0.10.0
the official script(TensorRT-LLM/examples/multimodal/run.py) use same prompt repeat to form a batch. but if I use different prompts to form a batch, the result is incorre…
-
Promethues metric locust_running is always zero during distributed load tests, any clues?
-
- **azure-batch**:
- **13.0.0**:
- **macOS 11.6**:
- **Python 3.10.8**:
**Describe the bug**
Once i try to create new batch task that depends on another one, I am getting the following error…
-
In cases where an incorrect R code is present, this causes an error with eggd_batch. The process for handling this error is to manually remove the sample from the manifest and then rerun eggd_batch. M…
-
Hi,
Thank you for the great work you've done on this model! Is there any way to batch the model using `funasr`? I've been trying to batch with padding and set the `padding_mask` to mask out the un…
-
Hello, when I run the training code python3 train.py --config ./config/detectiondiffusion.py, it shows:
(affpose) henry@yh:~/project/Language-Conditioned-Affordance-Pose-Detection-in-3D-Point-Cloud…
-
### Motivation.
A rework of https://github.com/vllm-project/vllm/issues/6226
After discussing further with the community, we find that the common use case for beam search is:
1. throughput orie…
-
### System Info
Python 3.11.9
transformers==4.40.2
peft==0.11.2
### Who can help?
@BenjaminBossan
I'm interested in using [Inference with different LoRA adapters in the same batch
](https://hu…
-
### The purpose and use-cases of the new component
This component is an experimental processor, forked from the [core
OpenTelemetry Collector `batchprocessor`
component](https://github.com/open-t…
-
### System Info
L4 GPU (AWS G6.12xl) with TensorRTLLM 0.11.0, running with Tritonbackends
### Who can help?
_No response_
### Information
- [ ] The official example scripts
- [ ] My own modified …