Closed fursovia closed 1 year ago
@ayoub-louati please take a look)
Sorry for being late, i'll test it very soon. @fursovia Thanks for this PR.
@fursovia Thank you very much for this fix and i'm really sorry for being late reviewing your PR, I was on other project.
Some models don't use
token_type_ids
in the forward pass. E.g.deberta
hastype_vocab_size=0
as a default value.What happens is the model ignores
token_type_ids
(https://github.com/huggingface/transformers/blob/bac2d29a802803a7f2db8e8597a2ec81730afcc9/src/transformers/models/deberta/modeling_deberta.py#L810)However, tokenizer doesn't know about this and
token_type_ids
is still intokenizer.model_input_names
.This mismatch leads to
And the triton inference server fails with
The proposed solution fixes this bug