mlc-ai / web-llm

High-performance In-browser LLM Inference Engine
https://webllm.mlc.ai
Apache License 2.0
13.79k stars 887 forks source link

Service worker engine hangs forever if client is lost while streaming results #620

Open Bainainai opened 1 month ago

Bainainai commented 1 month ago

If you close a page window while that page is streaming results from the service worker, the service worker becomes permanently unable to process any further results. To fix this, I think the service worker engine needs to detect when the requesting client is lost and end the request.