🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
133.93k
stars
26.79k
forks
source link
(Willing to PR) Make `tokenizer.padding_side` an argument instead of only being a field #30447
Open
fzyzcjy opened 6 months ago
Feature request
Hi thanks for the library! When using tokenizer, for example, for batch-generation with GPT2 (in https://discuss.huggingface.co/t/batch-generation-with-gpt2/1517), it seems that currently I have to do something like:
Therefore, it would be great to have:
just like what we do today for many options like
padding_strategy
etc.Motivation
(see above)
Your contribution
Yes, I am willing to PR