Switch to Lparallel and use conditions to cancel threads - Githubissues

atlas-engineer / prompter

Live-narrowing, fuzzy-matching, extensible prompt framework.

BSD 3-Clause "New" or "Revised" License

13 stars 1 forks source link

Switch to Lparallel and use conditions to cancel threads #2

Open Ambrevar opened 1 year ago

Ambrevar commented 1 year ago

The current architecture Calispel and with thread spwaning / destruction has some limits:

ECL (and maybe other implementations?) is not supported.
It may induce some race conditions (in particular with CCL), resulting in dangling threads.
It may be slow to spawn that many threads.
It results in a complicated API.

Suggestion: Switch to Lparallel with 1 kernel per prompt, and use conditions to interrupt.

Should fix ECL support.
Race condition surface dramatically reduced.
Should be faster since Lparallel managers its own workers, thus does not need to spawn a crazy number of threads.
Simpler API since the user does not need to know about the interruption channel and can use conditions instead.