Closed ggbetz closed 6 months ago
Check upon issue creation:
Parameters:
NEXT_MODEL_PATH=01-ai/Yi-34B-Chat NEXT_MODEL_REVISION=main NEXT_MODEL_PRECISION=bfloat16 MAX_LENGTH=2048 GPU_MEMORY_UTILIZATION=0.8 VLLM_SWAP_SPACE=8
ToDos:
Finished from my side but took quite a while (around 21 hours), just FYI.
Check upon issue creation:
Parameters:
ToDos: