Closed ggbetz closed 5 months ago
Check upon issue creation:
Parameters:
NEXT_MODEL_PATH=01-ai/Yi-1.5-34B-Chat NEXT_MODEL_REVISION=main NEXT_MODEL_PRECISION=bfloat16 MAX_LENGTH=2048 GPU_MEMORY_UTILIZATION=0.8 VLLM_SWAP_SPACE=4
ToDos:
completed, thx @yakazimir
Check upon issue creation:
Parameters:
ToDos: