Closed ggbetz closed 3 months ago
Check:
Parameters:
NEXT_MODEL_PATH=mistralai/Mixtral-8x7B-v0.1 NEXT_MODEL_REVISION=main NEXT_MODEL_PRECISION=bfloat16 MAX_LENGTH=2048 GPU_MEMORY_UTILIZATION=0.8 VLLM_SWAP_SPACE=4
Check:
Parameters: