search

microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.49k stars 2.9k forks source link

[web/js] need for more methods on tensor object #17372

Open guschmue opened 1 year ago

guschmue commented 1 year ago

original mentioned here: https://github.com/microsoft/onnxruntime/issues/17167

Methods that are commonly used by users are here: https://github.com/xenova/transformers.js/blob/main/src/utils/tensor.js and here: https://github.com/dakenf/stable-diffusion-webgpu-minimal/blob/main/src/lib/Tensor.ts

We will take a look what can be done.

fyi @dakenf, @fs-eire, @xenova

dakenf commented 1 year ago

Maybe worth mentioning that i thought JS operations with tensors are slow but sampling for stable diffusion takes about 10ms with this code https://github.com/dakenf/stable-diffusion-webgpu-minimal/blob/main/src/lib/schedulers/PNDMScheduler.ts

I haven't measured time spent to download the result from GPU, pass it back to JS and then upload back, so keeping tensors on GPU might be a good idea. But not sure if it's worth the effort though