search

airockchip / rknn-llm

Other
347 stars 29 forks source link

Need a way to stop inference midway #35

Closed Tidus1991 closed 4 months ago

Tidus1991 commented 5 months ago

Could we add a feature to stop an ongoing inference? It would be really useful for interactive applications where user inputs can change quickly. thanks.

waydong commented 4 months ago

v1.0.1 already supports interrupt inference, you can have a try.