logikon-ai / cot-eval

A framework for evaluating the effectiveness of chain-of-thought reasoning in language models.
https://huggingface.co/spaces/logikon/open_cot_leaderboard
MIT License
12 stars 2 forks source link

Evaluate: 01-ai/Yi-34B-Chat #33

Closed ggbetz closed 6 months ago

ggbetz commented 8 months ago

Check upon issue creation:

Parameters:

NEXT_MODEL_PATH=01-ai/Yi-34B-Chat
NEXT_MODEL_REVISION=main
NEXT_MODEL_PRECISION=bfloat16
MAX_LENGTH=2048 
GPU_MEMORY_UTILIZATION=0.8
VLLM_SWAP_SPACE=8

ToDos:

yakazimir commented 6 months ago

Finished from my side but took quite a while (around 21 hours), just FYI.