As I understand it, when I call the build_transformers_prefix_allowed_tokens_fn function, a TokenEnforcer instance is initialized to maintain the parsing state of a sequence:
Besides, in the huggingface transformers codes, this function is used to deal with multiple sequences in one batch, as in the following picture:
My question is that, does it mean that one TokenEnforcer instance can deal with multiple sequences in a batch, and maintain all the parsing state of the sequences? Will the sequences in one batch affect or mess up each other's parsing states?
As I understand it, when I call the
build_transformers_prefix_allowed_tokens_fnfunction, aTokenEnforcerinstance is initialized to maintain the parsing state of a sequence:Besides, in the huggingface transformers codes, this function is used to deal with multiple sequences in one batch, as in the following picture:
My question is that, does it mean that one
TokenEnforcerinstance can deal with multiple sequences in a batch, and maintain all the parsing state of the sequences? Will the sequences in one batch affect or mess up each other's parsing states?