Fix index out of bound error if input size is greater than the maximu…

mozilla-ai / lm-buddy

Your buddy in the (L)LM space.

Apache License 2.0

64 stars 3 forks source link

Closed imtihan closed 2 months ago

imtihan commented 3 months ago

…m context length

parameter value on tokenization

run an lm-buddy hf-evaluate job with the pegasus-xsum model and email thread summary (https://huggingface.co/datasets/sidhq/email-thread-summary) dataset. It should run without errors.

aittalam commented 2 months ago

Addressed in PR113