Closed ggbetz closed 6 months ago
Check:
Parameters:
NEXT_MODEL_PATH=meta-llama/Llama-2-70b-chat-hf NEXT_MODEL_REVISION=main NEXT_MODEL_PRECISION=float16 MAX_LENGTH=2048 GPU_MEMORY_UTILIZATION=0.8 VLLM_SWAP_SPACE=16
complete and published
Check:
Parameters: