search

ngxson / wllama

WebAssembly binding for llama.cpp - Enabling on-browser LLM inference
https://huggingface.co/spaces/ngxson/wllama
MIT License
444 stars 23 forks source link

Feature request: add a built-in way to interrupt inference #19

Closed flatsiedatsie closed 6 months ago

flatsiedatsie commented 6 months ago

It is possible to create some generic implementation to abort the inference?

In my hacky version of Wllama I had added a simple check to interrupt the inference for-loop if an 'interrupt' variable in the object's root was set to true.