Open kripper opened 1 year ago
First, congratulations for integrating the CTranslate2 implementation with HF Transformers. This was necessary.
Question: Is there any way to stop generate() when reaching a specific text sequence? We need this to deal with this issue: https://huggingface.co/clibrain/Llama-2-7b-ft-instruct-es/discussions/1
Somehing like a callback to cancel generate() would also be usefull (when a user triggers a cancel).
First, congratulations for integrating the CTranslate2 implementation with HF Transformers. This was necessary.
Question: Is there any way to stop generate() when reaching a specific text sequence? We need this to deal with this issue: https://huggingface.co/clibrain/Llama-2-7b-ft-instruct-es/discussions/1
Somehing like a callback to cancel generate() would also be usefull (when a user triggers a cancel).