search

OpenGenerativeAI / llm-colosseum

Benchmark LLMs by fighting in Street Fighter 3! The new way to evaluate the quality of an LLM
https://huggingface.co/spaces/junior-labs/llm-colosseum
MIT License
1.34k stars 160 forks source link

NEW DOCKER IMAGE ERROR: fighter starts, but two Kens do not fight. #53

Closed taozhiyuai closed 7 months ago

taozhiyuai commented 7 months ago

fighter starts, but two Kens do not fight. @oulianov ' (0)Restarting system and (optionally) setting difficulty 🏟️ (864b) (0)Starting game 🏟️ (864b) (0)Waiting for fight to start Exception in thread Thread-5: Exception in thread Thread-6: Traceback (most recent call last): File "/Users/taozhiyu/miniconda3/envs/streetfighter/lib/python3.9/threading.py", line 980, in _bootstrap_inner Traceback (most recent call last): File "/Users/taozhiyu/miniconda3/envs/streetfighter/lib/python3.9/threading.py", line 980, in _bootstrap_inner self.run() File "/Users/taozhiyu/Downloads/llm-colosseum/eval/game.py", line 369, in run self.run() File "/Users/taozhiyu/Downloads/llm-colosseum/eval/game.py", line 383, in run self.game.player_2.robot.plan() File "/Users/taozhiyu/Downloads/llm-colosseum/agent/robot.py", line 134, in plan self.game.player_1.robot.plan() File "/Users/taozhiyu/Downloads/llm-colosseum/agent/robot.py", line 134, in plan next_steps_from_llm = self.get_moves_from_llm() File "/Users/taozhiyu/Downloads/llm-colosseum/agent/robot.py", line 293, in get_moves_from_llm next_steps_from_llm = self.get_moves_from_llm() File "/Users/taozhiyu/Downloads/llm-colosseum/agent/robot.py", line 293, in get_moves_from_llm llm_stream = self.call_llm() File "/Users/taozhiyu/Downloads/llm-colosseum/agent/robot.py", line 369, in call_llm llm_stream = self.call_llm() File "/Users/taozhiyu/Downloads/llm-colosseum/agent/robot.py", line 369, in call_llm resp = client.stream_chat(messages) AttributeError: 'NoneType' object has no attribute 'stream_chat' resp = client.stream_chat(messages) AttributeError: 'NoneType' object has no attribute 'stream_chat' '

oulianov commented 7 months ago

This sounds like a duplicate issue of https://github.com/OpenGenerativeAI/llm-colosseum/issues/50

taozhiyuai commented 7 months ago

latest version

    player_1=Player1(  
        nickname="Baby",  
        model="ollama:gemma:7b-instruct-q8_0",
    ),
    player_2=Player2(  
        nickname="Daddy",  
        model="ollama:llama3:8b-instruct-q8_0",

before I upgrade docker image, it works fine.

taozhiyuai commented 7 months ago

Done!

kkur0same commented 6 months ago

hi @taozhiyuai , I encountered the same issues but using Mistral AI's API. Can you share how you resolved this?