-
## What are the problems?(screenshots or detailed error messages)
想问下有性能分析的工具嘛？profiler相关，还是只能用nsight profile这种自己去看一些算子性能
## What are the types of GPU/CPU you are using?
GPU：A100-80G-SXM4
## What…
-
I've done the following:
> Alternatively, one may also skip the quantization process and directy download the quantized VILA-1.5 checkpoints from [here](https://huggingface.co/Efficient-Large-Model…
-
## Describe the bug
Unable to open AI playground.. It hang at loading screen.
Tried installed the Latest Microsoft Visual C++ Redistributable Version - The latest version is 14.40.33816.0
No Pytho…
-
**Is your feature request related to a problem? Please describe:**
I would like to run this in a docker container to get the obvious benefits of containers.
**Describe the solution you'd like:**…
-
I'm running the tutorial [vllm/offline_inference_with_prefix.py](https://github.com/vllm-project/vllm/blob/main/examples/offline_inference_with_prefix.py) and measuring the generation times, again bel…
-
Currently [HUGGINGFACE_OFFLINE=1](https://github.com/shalb/charts/blob/7c29f2185336ed8d9cb14ffae9942f6b95462d12/huggingface-model/templates/application.yaml#L101-L102) is hardcoded in the helm templat…
slyt updated
3 months ago
-
The newest drivers are in use, the system is a Ryzen 2700x CPU with 16GB of RAM and a 16GB A770 GPU on Windows 11.
The instructions in the docs were followed precisely.
Upon attempting to execut…
-
This is highly speculative in terms of usefulness, and the UI would need to be considered carefully. Use case would be for summarizing articles retrieved from the ZIM. Over time, it might be possible …
-
**Is your feature request related to a problem? Please describe.**
We are able to infer the recommendation by the qualification tool but the recommendation is based on vague GPUs.
In recent experi…
-
### Your current environment
The output of `python collect_env.py`
```text
python collect_env.py
Collecting environment information...
PyTorch version: 2.4.0+cu121
Is debug build: False
C…