search

delphi-suite / delphi

small language models training made easy
Apache License 2.0
8 stars 1 forks source link

pad token is set to 0 in `generation_config.json` #132

Closed jettjaniak closed 1 month ago

jettjaniak commented 2 months ago

Our tokenizer doesn't have a pad token (maybe a mistake)

model.generate()
A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer.
jettjaniak commented 2 months ago

I'm thinking we should add pad token and retrain the tokenizer