NVIDIA / NeMo-Aligner

Scalable toolkit for efficient model alignment

Apache License 2.0

620 stars 78 forks source link

fix: pad_id=None in tokenizer no longer errors for TRTLLM generation #288

Closed terrykong closed 1 month ago

terrykong commented 2 months ago

What does this PR do ?

Fixes a crash that occurs when using llama3 tokenizers since they don't have a pad_id in their huggingface tokenizer.

The fix is to use an out of bounds id if there's no pad_id, then swap it with eos_id right after generation. Credit to @gshennvm for the idea

Changelog

Please update the CHANGELOG.md under next version with high level changes in this PR.

Usage

You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

[ ] Make sure you read and followed Contributor guidelines
[ ] Did you write any new necessary tests?
[ ] Did you add or update any necessary documentation? Make sure to also update the NeMo Framework User Guide which contains the tutorials

Checklist when contributing a new algorithm

[ ] Does the trainer resume and restore model state all states?
[ ] Does the trainer support all parallelism techniques(PP, TP, DP)?
[ ] Does the trainer support max_steps=-1 and validation?
[ ] Does the trainer only call APIs defined in alignable_interface.py?
[ ] Does the trainer have proper logging?

Additional Information

Related to # (issue)