Closed sebastian-nehrdich closed 2 weeks ago
Hi @sebastian-nehrdich 👋
Thanks for reporting this. After the log message Shard complete standard error output
is there any exit code from the shard logged as well?
This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.
System Info
This is the current version installed from github, text-generation-launcher 2.1.1-dev0
Information
Tasks
Reproduction
text-generation-launcher --max-input-tokens 1024 --max-total-tokens 2048 --max-batch-size 12 -p 3409 --model-id google/gemma-2-9b --master_port 29501
Expected behavior
The model should start without the error message.