search

elixir-nx / ortex

ONNX Runtime bindings for Elixir
MIT License
122 stars 15 forks source link

Implement slice on Ortex.Backend #8

Closed mortont closed 1 year ago

mortont commented 1 year ago

When sending Nx.Serving.batched_run batches from multiple processes to an Ortex.Serving process, the run fails with:

    ** (RuntimeError) operation slice is not yet supported on Ortex.Backend.
        (ortex 0.1.3) lib/ortex/backend.ex:106: Ortex.Backend.slice/5
        (nx 0.5.3) lib/nx/defn/composite.ex:101: anonymous fn/3 in Nx.Defn.Composite.traverse/2
        (elixir 1.14.0) lib/enum.ex:1780: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
        (nx 0.5.3) lib/nx/container.ex:110: Nx.Container.Tuple.traverse/3
        (nx 0.5.3) lib/nx/defn/composite.ex:101: Nx.Defn.Composite.traverse/2
        (nx 0.5.3) lib/nx/serving.ex:682: Nx.Serving.receive_batched/6
        (nx 0.5.3) lib/nx/serving.ex:648: Nx.Serving.local_batched_run/3
        (nx 0.5.3) lib/nx/serving.ex:618: Nx.Serving.local_batched_run!/3

This is due to Ortex.Backend not implementing slice. This will need to be implemented for Ortex.Serving to be complete.