Closed ggbetz closed 4 days ago
For {XX} in [0.5B, 1.8B, 4B, 7B, 14B, 32B and 72B]:
{XX}
Check upon issue creation:
Parameters:
NEXT_MODEL_PATH=Qwen/Qwen1.5-XX-Chat NEXT_MODEL_REVISION=main NEXT_MODEL_PRECISION=bfloat16 MAX_LENGTH=2048 GPU_MEMORY_UTILIZATION=0.7 VLLM_SWAP_SPACE=8
ToDos:
Qwen models fail to generate reasoning traces. https://github.com/logikon-ai/cot-eval/blob/f9bfe8f757edbed49324df680214a24fbde37213/src/cot_eval/__main__.py#L139C1-L146C53
For
{XX}
in [0.5B, 1.8B, 4B, 7B, 14B, 32B and 72B]:Check upon issue creation:
Parameters:
ToDos: