[Feature Request] DFT/STFT WebGPU op support (web/js)

microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

https://onnxruntime.ai

MIT License

13.48k stars 2.76k forks source link

[Feature Request] DFT/STFT WebGPU op support (web/js) #20997

Open xenova opened 3 weeks ago

xenova commented 3 weeks ago

Describe the feature request

To add WebGPU support for DFT and STFT ops:

https://onnx.ai/onnx/operators/onnx__STFT.html
https://onnx.ai/onnx/operators/onnx__DFT.html

Currently, neither are supported yet (as stated here).

Describe scenario use case

This will be used to optimize preprocessing performance for audio models like Whisper, specifically to be used in Transformers.js.

xenova commented 3 weeks ago

cc @guschmue for this. Another thing I've noticed is that the WASM version is much slower than a pure JS implementation (which I currently use). So, might be something to look into too.

guschmue commented 3 weeks ago

makes sense. Kind surprising that wasm is slower then javascript, maybe it's an algo choice. We'll look at it.