elixir-nx / bumblebee

Pre-trained Neural Network models in Axon (+ 🤗 Models integration)
Apache License 2.0
1.33k stars 96 forks source link

Don't block on tensor access in postprocessing #245

Closed jonatanklosko closed 1 year ago

jonatanklosko commented 1 year ago

Follow up to #244.

Accessing the whole binary with Nx.to_flat_list or Nx.backend_transfer is not blocked by ongoing computation, however tensor access involves slicing and therefor does block. So whenever we need to slice, we are going to transfer to binary backend first.