Closed mortont closed 1 year ago
When sending Nx.Serving.batched_run batches from multiple processes to an Ortex.Serving process, the run fails with:
Nx.Serving.batched_run
Ortex.Serving
** (RuntimeError) operation slice is not yet supported on Ortex.Backend. (ortex 0.1.3) lib/ortex/backend.ex:106: Ortex.Backend.slice/5 (nx 0.5.3) lib/nx/defn/composite.ex:101: anonymous fn/3 in Nx.Defn.Composite.traverse/2 (elixir 1.14.0) lib/enum.ex:1780: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3 (nx 0.5.3) lib/nx/container.ex:110: Nx.Container.Tuple.traverse/3 (nx 0.5.3) lib/nx/defn/composite.ex:101: Nx.Defn.Composite.traverse/2 (nx 0.5.3) lib/nx/serving.ex:682: Nx.Serving.receive_batched/6 (nx 0.5.3) lib/nx/serving.ex:648: Nx.Serving.local_batched_run/3 (nx 0.5.3) lib/nx/serving.ex:618: Nx.Serving.local_batched_run!/3
This is due to Ortex.Backend not implementing slice. This will need to be implemented for Ortex.Serving to be complete.
Ortex.Backend
slice
When sending
Nx.Serving.batched_run
batches from multiple processes to anOrtex.Serving
process, the run fails with:This is due to
Ortex.Backend
not implementingslice
. This will need to be implemented forOrtex.Serving
to be complete.