I notice that ctransformers.generate accepts this kwarg, but ignores it.
This resulted in a sneaky segfault since this library isn't threadsafe. I thought model generation was stopped, but it actually wasn't, so upon a second generation, it would segfault mysteriously.
Huggingface transformers offers
stopping_criteria
: https://huggingface.co/transformers/v4.6.0/_modules/transformers/generation_stopping_criteria.htmlI use this with a
threading.Event
so I can stop generation from a separate thread, and it works great withtransformers
:I notice that
ctransformers.generate
accepts this kwarg, but ignores it.This resulted in a sneaky
segfault
since this library isn't threadsafe. I thought model generation was stopped, but it actually wasn't, so upon a second generation, it would segfault mysteriously.