-
From twitter: SFG l’espérance, Saint Martin collective of FranceRelieved face, thé other one on st Martin/st Maarten (SXM)
-
## 🐛 Bug
Hi pytorch teams,
When increasing the inference batch size on A100 machine, the model output will differ than that of batch size = 1.
I simplify the model to only one torch.nn.Conv2d lay…
-
cm run script --tags=run-mlperf,inference,_performance-only,_short
--division=open
--category=network
--device=c…
-
Found some differences in the polarization phases while testing SHADOW4 in a scanning X-ray microscope workspace: [SXM-workspace](https://github.com/oasys-kit/shadow4workspaces/blob/main/reyes_id21_SX…
-
Thank you for your hard work.
The performance difference between A100 and H100 is not significant. I used the official VLLM image 0.2.4 on Docker Hub.
I set the prompt and completion to 500, and…
-
github.com/hugomrdias/concat-stream.git Repository not found
-
Hello,
I am using Nvidia A100 160gb GPU and just testing different mining software. (Don't predict that it's commercial product that I am using it personally -- as it's my own box that I have manag…
nv160 updated
2 years ago
-
### System Info
- tensorrtllm_backend built using Dockerfile.trt_llm_backend
- main branch tesnorrt llm (0.13.0.dev20240813000)
- 8xH100 SXM
- Driver Version: 535.129.03
- CUDA Version: 12.5
…
-
Hi,
I am trying to set up vLLM Mixtral 8x7b on GCP. I have a VM with two A100 80GBs, and am using the following setup:
docker image: vllm/vllm-openai:v0.3.0
Model: mistralai/Mixtral-8x7B-Instruct…
-
### 🐛 Describe the bug
I came across an error caused by a TorchScript model after two iterations of forward-backward passes.
The following script can reproduce the error.
The serialized model can…