-
Explaining and demonstrating the use of tpot library that can be used to find the best model with the best parameters for classification and regression task without much efforts.
Please assign this…
-
### Your current environment
The output of `python collect_env.py`
```text
Collecting environment information...
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTor…
-
I did some tests in order to find better parameter to speed up, and it appears that there hasn't been a significant change in TTFT (Time To First Token). Is my TTFT correct? I feel it might be a bit t…
-
For an Open submission, we have tried setting TTFT/TPOT latencies in `user.conf` e.g.:
mixtral-8x7b.Server.ttft_latency = 3000
mixtral-8x7b.Server.tpot_latency = 300
However, in the resulting…
-
### Your current environment
v0.5.2. vLLM env is not an issue so I will just skip the collection process
### 🐛 Describe the bug
I am running benchmark tests and notice one potential problem. …
-
I've run the benchmark_genai.py for CPU, GPU, NPU on MTL U9, here is the logs:
(env_ov_genai) c:\AIGC\openvino\openvino.genai\samples\python\benchmark_genai>python benchmark_genai.py -m c:\AIGC\openv…
-
### Anything you want to discuss about vllm.
I am profiling TTFT and TPOT on my machine, I could not explain the behavior of TTFT thus opened this issue to seek for advice.
Below figure shows the …
-
When I try to run it through Windows on the Docker machine, it gives this error. However, I updated the python I'm running. Python is currently version 3.11.4 and still presents this error. The docker…
-
Use teapot feature selection strategy on tsflex generated features.
-
A few users reported mapping errors on a few integrations. We suspect these problems may be related to integrations that migrated to ecs@mappings with recent updates.
Here is the list of fields with …