search

elixir-nx / bumblebee

Pre-trained Neural Network models in Axon (+ 🤗 Models integration)
Apache License 2.0
1.33k stars 96 forks source link

Align batch handling for serving run and batched run #252

Closed jonatanklosko closed 1 year ago

jonatanklosko commented 1 year ago

Uses the new Nx.Serving.batch_size/2 and also converts the input to a basic stream.