philschmid / deep-learning-pytorch-huggingface

MIT License
618 stars 143 forks source link

Add tokenizer to Trainer object #1

Open mrm8488 opened 1 year ago

mrm8488 commented 1 year ago

This way, It will be pushed too on trainer.push_to_hub()

philschmid commented 1 year ago

I kept its purpose separate if you want to "decouple" the training and processing parts.