logikon-ai / cot-eval

A framework for evaluating the effectiveness of chain-of-thought reasoning in language models.
https://huggingface.co/spaces/logikon/open_cot_leaderboard
MIT License
5 stars 1 forks source link

Evaluate: internlm/internlm2-XX #41

Closed ggbetz closed 2 months ago

ggbetz commented 3 months ago

For XX in [7B, 20B, Chat-7B, Chat-20B]:

Check upon issue creation:

Parameters:

NEXT_MODEL_PATH=internlm/internlm2-chat-7b
NEXT_MODEL_REVISION=main
NEXT_MODEL_PRECISION=bfloat16
MAX_LENGTH=2048 
GPU_MEMORY_UTILIZATION=0.7
VLLM_SWAP_SPACE=8

ToDos: