search

huggingface / candle

Minimalist ML framework for Rust
Apache License 2.0
14.68k stars 839 forks source link

ONNX: Gather for tensors of rank > 0. #1305

Open xd009642 opened 8 months ago

xd009642 commented 8 months ago

Firstly thanks for candle, it's an impressive project and the speed at which updates are coming is really cool as a user!

I'm trying to use a speedyspeech onnx model in it right now and there's a failing op:

not yet implemented: implement gather for Tensor[dims 78, 128; f32] Tensor[dims 1, 16; i64] axis 0

I can provide my ONNX or assist via PRs if required (may need some pointers). Otherwise just wondering if there's any idea when Gather might fully land or if there are any difficulties in the implementation that might mean I should look for an alternative solution in the near term.

LaurentMazare commented 8 months ago

Ah, the ONNX bits are brand new and only support a subset of the operations. It would certainly be helpful if you can share the ONNX file that you tried to run so that we can see if we can make it pass all ops.

xd009642 commented 8 months ago

Sure here's the model speedyspeech.zip

smallbraingames commented 2 weeks ago

did you ever get this running?

xd009642 commented 2 weeks ago

Not speedyspeech but whatever candle does it won't work with that exported model. Not even onnxruntime works because the version of torch they used can generate invalid onnx graphs that nothing can run

smallbraingames commented 2 weeks ago

ah got it, thank you

I wonder if there's a cheaper tts in rust than piper (which doesn't run in candle as well but does in ort)

anyways, thanks