Closed ggbetz closed 7 months ago
Check:
Parameters:
NEXT_MODEL_PATH=01-ai/Yi-34B NEXT_MODEL_REVISION=main NEXT_MODEL_PRECISION=bfloat16 MAX_LENGTH=2048 GPU_MEMORY_UTILIZATION=0.8 VLLM_SWAP_SPACE=8
Check:
Parameters: