not set padding tokens as it is a problem within e.g. LLAMA 3 8B Instruct. We set it per default to EOS token as this is an appropriate default token in inference.
Additionally we offer setting of system props over arg as it influences models results and is petter an arg and not a hardcoded string
thank you for your efforts.
kindest regards
Carsten
hi @Psycoy This PR handles:
thank you for your efforts. kindest regards Carsten