A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer.

minimaxir / aitextgen

A robust Python tool for text-based AI training and generation using GPT-2.

https://docs.aitextgen.io

MIT License

1.84k stars 218 forks source link

A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer. #197

Open Blazeolmo opened 1 year ago

Blazeolmo commented 1 year ago

What ze hell does that mean?

kiasar commented 1 year ago

My question too!

Choitsugun commented 1 year ago

That is used to solve the problem of unaligned generation caused by padding during the batch inference of the decoder-only model.

Vectorrent commented 1 year ago

I think this must be a hard-coded warning, coming from the upstream Transformers package. No matter how hard I've tried, I can't seem to suppress the message.