-
Like what we did for training acceleration:
https://github.com/Project-MONAI/tutorials/blob/main/acceleration/fast_training_tutorial.ipynb
-
### Prerequisites
- [X] I am running the latest code. Mention the version if possible as well.
- [X] I carefully followed the [README.md](https://github.com/ggerganov/llama.cpp/blob/master/README.md)…
-
ERROR: MethodError: no method matching ParametricMCPs.ParametricMCP(::MCPGameSolver.var"#38#54"{…}, ::ParametricMCPs.SparseFunction{…}, ::ParametricMCPs.SparseFunction{…}, ::Vector{…}, ::Vector{…}, ::…
-
Hi! Very impressive project!
My main goal is to export the model to intermediate format and test accelerability on many platforms. I am trying to accelerate the assembled convolution module for be…
-
Hello, I am currently using the AMD Instinct MI50 GPU to train models. It has 26 Tflops of fp16 and 13 Tflops of fp32 compute power, but it lacks tensor cores.
My experiments on PyTorch indicate th…
-
Can we support NPU acceleration library, NPU inference model save/load in low bits?
It takes about 48s to load the 7B model directly.
-
My use case is inference acceleration on a CPU using TensorFlow Serving, and my hardware architecture is AArch64 (ARMv8). Currently, I've noticed that with oneDNN enabled, the performance bottleneck i…
-
Hi! Thank you so much for such an awesome repository!
The torch modules here are executed in `inference_mode` not `no_grad`, which causes some problems when doing some accelerations, such as torch.…
-
**Bug Description**
The ONNX CUDA session is not working in the Python backend. When attempting to run inference using the ONNX model with CUDAExecutionProvider, the session fails to initialize or ex…
-
Hi!
I heard about a very promising model some while ago that you might be interested in. It's called fish.audio.
Here's a youtube demo : https://www.youtube.com/watch?v=Ghc8cJdQyKQ
Here's the…