search

jina-ai / jerboa

LLM finetuning
Apache License 2.0
41 stars 4 forks source link

fix: enforce eos token id during generation and skip special tokens when decoding #94

Closed alaeddine-13 closed 1 year ago

alaeddine-13 commented 1 year ago

closes: https://github.com/jina-ai/jerboa/issues/72