Oufattole / meds-torch

MIT License
16 stars 2 forks source link

The EOS max seq length adjustment should happen in the dataset, not in the config, to avoid violating the user's inputs. #99

Closed mmcdermott closed 1 month ago

mmcdermott commented 1 month ago

If the user sets a certain max seq length, no more than that many tokens should be passed to the GPU.