search

xenova / transformers.js

State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!
https://huggingface.co/docs/transformers.js
Apache License 2.0
10.09k stars 591 forks source link

GPU Acceleration to increase performance #377

Open milind-yadav opened 8 months ago

milind-yadav commented 8 months ago

Do we have any option to use GPU to increase performance of model loading and detection? As currently in Object Detection it's taking around 10 seconds. If we want to do this on GPU, can we do that?

Running below lines through web worker, increases overall UI experience but not increases any performance.

const model = await pipeline("object-detection", "Xenova/detr-resnet-50");
const result = await model(img, { threshold: 0.9 });

Can we use GPU for that?

felladrin commented 8 months ago

We can't use GPU yet, but keep an eye on this other issue, which relates to the progress of WebGPU support: