Closed jparismorgan closed 9 months ago
@jparismorgan This behavior is intentional and follows the JS AbortController
convention.
If your intent is to ignore an aborted evaluation then you should just detect this kind of error and ignore it.
For example:
import {AbortError, ...} from "node-llama-cpp";
...
try {
await session.prompt(message, {
onToken,
signal: abortController.signal
});
} catch(err) {
if (!(err instanceof AbortError))
throw err;
}
Issue description
When aborting a
prompt()
an error is thrown.Expected Behavior
I can abort a generation.
Actual Behavior
This is thrown:
which when I click in leads to this line:
and before that, this line:
Steps to reproduce
I first set up things like this:
Then I kick off a text generation:
And then I abort it:
My Environment
node-llama-cpp
versionAdditional Context
Is the appropriate thing to do here just to wrap the code in a try catch? Or is this actually a bug? Thank you for any help!
Relevant Features Used
Are you willing to resolve this issue by submitting a Pull Request?
Yes, I have the time, but I don't know how to start. I would need guidance.