I use transformers pipeline to generate json dictionaries and I need to specify a prefix_allowed_tokens_fn such that the tokens that can be generated at some steps are fixed. By looking into the source code, it doesn't seem like the deepspeed pipeline support this. Could someone verify whether it is supported please？
I use transformers pipeline to generate json dictionaries and I need to specify a prefix_allowed_tokens_fn such that the tokens that can be generated at some steps are fixed. By looking into the source code, it doesn't seem like the deepspeed pipeline support this. Could someone verify whether it is supported please？