-
I have followed the Tensorflow2 documentation to convert my trained tf.estimator model to tflite model; in order to convert my model, first I had to save my model in saved_model format with a input_r…
-
While testing out the batched inference on openllama, I notice that for a single prompt it takes 2.56 seconds but for 8 prompts it takes 24.62 secs. Essentially no improvement in performance.
I mig…
-
Every so often I have found my tensorflow server killing itself due to memory usage hitting capacity. I can't quite figure out how or why this happening because it only happens occasionally and my pro…
-
It is quite common to use shape polymorphism in TFLite (for instance when processing audio), yet our [mnist example](https://github.com/google/jax/tree/main/jax/experimental/jax2tf/examples/tflite/mni…
-
## Description
Hi, TensorRT team. Thank you for your excellent work. I am benefiting from the stable diffusion demo. This script significantly speeds up my inference serving.
I encountered some OO…
-
### Description
I have a simple model that I've been able to run on the deb board that utilizes the edge TPU and have benchmarked it's performance running it 1000 times and averaging. It's getting ~3…
-
I will progressively summarize talks I find illuminating from the [Stanford MLSys](https://mlsys.stanford.edu/) Seminar Series here.
Talk Link: [https://www.youtube.com/watch?v=DB7oOZ5hyrE](https://w…
-
**System information**
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 20.04
- TensorFlow installed from (source or binary): source
- TensorFlow version (or github SHA if fr…
-
### 🚀 The feature
As far as I know, there are no examples or documentation on serving Speech2Text models from Huggingface, such as Wav2Vec2. How could I enable serving with Wav2Vec2 Huggingface pre-t…
-
Click to expand!
### Issue Type
Support
### Have you reproduced the bug with TF nightly?
No
### Source
binary
### Tensorflow Version
tf2.8
### Custom Code
Yes
### O…