search

xenova / transformers.js

State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!
https://huggingface.co/docs/transformers.js
Apache License 2.0
11.08k stars 684 forks source link

Support static shape #719

Open hans00 opened 5 months ago

hans00 commented 5 months ago

Feature request

Support static shape (auto padding to max length on input) and static shape KV cache for LLM.

Motivation

Static shape will be problem when enable NPU, WebNN or CoreML.

Your contribution

I can submit a PR. But I'm not pro of models, not sure correct implement of static KV cache.