-
Hi, @haileyschoelkopf Thank you for your awsome open-source work. We have been evaluating using `lm-eval` and noticed that when using `accelerate` for data parallel inference, the number of GPUs utili…
-
This issue will serve as a tracking mechanism for all the work being done to get FMS models working with torch.export().
The initial list of work we have identified is the following:
- We need a…
-
# ❓ Questions & Help
## Details
Hi, I have been experimenting with an existing TF2 model using the merlin-tensorflow image. This has allowed me to leverage the SOK toolkit for the SparseEmbeddin…
-
### Describe the issue
In a scenario where multiple GPU devices are available, when selecting the TensorrtExecutionProvider and choosing device_id = 0, the model infers perfectly. However, when usi…
dat58 updated
2 months ago
-
Hi,
I am trying to run the example script provided for llama model for inference only. Since the repository is going through migration and a lot of changes, I went back and install the stable `v0.2…
-
### System Info
infinity 0.0.53
OS version: linux
Model being used: dunzhang/stella_en_1.5B_v5
Hardware used: NVIDIA A100
### Information
- [ ] Docker
- [X] The CLI directly via pip
##…
-
### Describe the issue
We have a Flask-based API for running computer vision models (YOLO and classifiers) using ONNX Runtime. The models, originally trained in PyTorch, are converted to ONNX forma…
-
Hi,
is it possible to run PyTorch model inference (server) with go as well? any projects you know if?
Thanks
-
### 🐛 Describe the bug
Hello,
I would like to ask your help.
I am using KServe and would like to deploy a PyTorch model with it.
My problem is that I am getting models missing error messages…
-
Since the coqui docs recommend the use of `deepspeed` to speed up their XTTS model I wanted to give this a try.
To make it work I did the following:
- I had to rebuild pytorch with `USE_NCCL=1` be…
hslr4 updated
2 months ago