Closed yhyu13 closed 11 months ago
I tried gpt-3.5-turbo-16k with the docker meta-gpt:v.0.3.1 cli_snake example. it fails, but gpt4 succeeded.
It seems to me gpt3.5 has failed with code syntax check? Am I getting it wrong?
Below is the cmd output:
2023-09-09 14:17:27.444 | INFO | metagpt.config:init:44 - Config loading done. 2023-09-09 14:17:29.222 | INFO | metagpt.software_company:invest:39 - Investment: $3.0. 2023-09-09 14:17:29.222 | INFO | metagpt.roles.role:_act:166 - Alice(Product Manager): ready to WritePRD Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/tenacity/_asyncio.py", line 50, in call result = await fn(*args, kwargs) File "/app/metagpt/metagpt/actions/action.py", line 62, in _aask_v1 instruct_content = output_class(parsed_data) File "pydantic/main.py", line 341, in pydantic.main.BaseModel.init
The boss wants a command-line interface (CLI) snake game.
Ensure smooth gameplay and high performance.
As a user, I want to be able to start a new game or quit the game at any time.
Snake Game AI: An AI-powered snake game that can be played against computer-controlled opponents.
quadrantChart
title Reach and engagement of snake game variants
x-axis Low Reach --> High Reach
y-axis Low Engagement --> High Engagement
quadrant-1 Need to promote
quadrant-2 May be improved
quadrant-3 Re-evaluate
quadrant-4 We should expand
"Python Snake Game": [0.8, 0.9]
"CLI Snake Game": [0.6, 0.7]
"Snake Game App": [0.9, 0.8]
"Online Snake Game": [0.7, 0.6]
"Snake Game Console": [0.7, 0.7]
"Snake Game Website": [0.8, 0.6]
"Snake Game AI": [0.6, 0.8]
"Our Target Product": [0.5, 0.6]
The product should be a command-line interface (CLI) snake game that provides a fun and engaging gameplay experience. It should have intuitive controls, smooth gameplay, and high performance. The game should end when the snake hits the wall or its own body, and the player's score should be displayed on the screen. The game should also have a high score feature and allow the player to start a new game or quit at any time.
Allow the player to start a new game or quit at any time (P2)
The game will be displayed in the command line interface. The snake will be represented by a series of characters, such as "O", and the food will be represented by a different character, such as "*". The player will control the snake using arrow keys. The current score and high score will be displayed on the screen. The game will have a simple layout with the snake and food positioned within the boundaries of the command line window. The game will have a clean and minimalistic style, with clear and easy-to-read characters.
The boss wants a command-line snake game.
Ensure smooth and responsive gameplay.
As a user, I want to be able to pause and resume the game.
Snake Attack: A snake game with a twist, where the snake can attack other creatures.
quadrantChart
title Reach and engagement of snake games
x-axis Low Reach --> High Reach
y-axis Low Engagement --> High Engagement
quadrant-1 We should expand
quadrant-2 Need to promote
quadrant-3 Re-evaluate
quadrant-4 May be improved
"Python Snake Game": [0.3, 0.6]
"Command-line Snake": [0.45, 0.23]
"Snake Game Deluxe": [0.57, 0.69]
"Snake Mania": [0.78, 0.34]
"Snake.io": [0.40, 0.34]
"Snake Escape": [0.35, 0.78]
"Snake Attack": [0.6, 0.7]
"Our Target Product": [0.5, 0.6]
The product should be a command-line snake game with smooth and responsive gameplay. It should have a user-friendly interface and display the score and length of the snake on the screen. The game should end when the snake hits the wall or itself. It should also progressively increase in difficulty as the player scores more points. The game should have a pause and resume functionality.
Implement pause and resume functionality (P2)
The game will be displayed in the command-line interface. The snake will be represented by a character, such as an 'O', and the food will be represented by a different character, such as an 'X'. The score and length of the snake will be displayed at the top of the screen. The arrow keys will be used to control the snake's movement. The game will have a simple and minimalistic style, with clear and easy-to-read text. The layout will be organized, with the game area in the center of the screen and the score and length displayed at the top. pydantic.error_wrappers.ValidationError: 5 validation errors for prd Requirement Pool -> 0 value is not a valid tuple (type=type_error.tuple) Requirement Pool -> 1 value is not a valid tuple (type=type_error.tuple) Requirement Pool -> 2 value is not a valid tuple (type=type_error.tuple) Requirement Pool -> 3 value is not a valid tuple (type=type_error.tuple) Anything UNCLEAR field required (type=value_error.missing)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/metagpt/startup.py", line 36, in
Another try with gpt3.5-turbo which is 4096 token maxed, metagpt fails with token max out even though I set MAX_TOKEN in yaml to be 1500 which is the default option:
2023-09-09 14:32:04.840 | INFO | metagpt.config:init:44 - Config loading done. 2023-09-09 14:32:06.661 | INFO | metagpt.software_company:invest:39 - Investment: $3.0. 2023-09-09 14:32:06.661 | INFO | metagpt.roles.role:_act:166 - Alice(Product Manager): ready to WritePRD 2023-09-09 14:32:37.106 | INFO | metagpt.roles.role:_act:166 - Bob(Architect): ready to WriteDesign 2023-09-09 14:33:10.803 | INFO | metagpt.utils.mermaid:mermaid_to_file:40 - Generating /app/metagpt/workspace/cli_snake_game/resources/competitive_analysis.pdf.. 2023-09-09 14:33:11.884 | INFO | metagpt.utils.mermaid:mermaid_to_file:40 - Generating /app/metagpt/workspace/cli_snake_game/resources/competitive_analysis.svg.. 2023-09-09 14:33:12.930 | INFO | metagpt.utils.mermaid:mermaid_to_file:40 - Generating /app/metagpt/workspace/cli_snake_game/resources/competitive_analysis.png.. 2023-09-09 14:33:14.123 | INFO | metagpt.actions.design_api:_save_prd:110 - Saving PRD to /app/metagpt/workspace/cli_snake_game/docs/prd.md 2023-09-09 14:33:14.125 | INFO | metagpt.utils.mermaid:mermaid_to_file:40 - Generating /app/metagpt/workspace/cli_snake_game/resources/data_api_design.pdf.. 2023-09-09 14:33:15.291 | INFO | metagpt.utils.mermaid:mermaid_to_file:40 - Generating /app/metagpt/workspace/cli_snake_game/resources/data_api_design.svg.. 2023-09-09 14:33:16.437 | INFO | metagpt.utils.mermaid:mermaid_to_file:40 - Generating /app/metagpt/workspace/cli_snake_game/resources/data_api_design.png.. 2023-09-09 14:33:17.760 | INFO | metagpt.utils.mermaid:mermaid_to_file:40 - Generating /app/metagpt/workspace/cli_snake_game/resources/seq_flow.pdf.. 2023-09-09 14:33:18.946 | INFO | metagpt.utils.mermaid:mermaid_to_file:40 - Generating /app/metagpt/workspace/cli_snake_game/resources/seq_flow.svg.. 2023-09-09 14:33:20.042 | INFO | metagpt.utils.mermaid:mermaid_to_file:40 - Generating /app/metagpt/workspace/cli_snake_game/resources/seq_flow.png.. 2023-09-09 14:33:21.408 | INFO | metagpt.actions.design_api:_save_system_design:119 - Saving System Designs to /app/metagpt/workspace/cli_snake_game/docs/system_design.md 2023-09-09 14:33:21.411 | INFO | metagpt.roles.role:_act:166 - Eve(Project Manager): ready to WriteTasks 2023-09-09 14:33:45.424 | INFO | metagpt.actions.write_code:run:77 - Writing snake.py.. 2023-09-09 14:33:59.786 | INFO | metagpt.actions.write_code:run:77 - Writing food.py.. Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/tenacity/_asyncio.py", line 50, in call result = await fn(*args, kwargs) File "/app/metagpt/metagpt/actions/write_code.py", line 71, in write_code code_rsp = await self._aask(prompt) File "/app/metagpt/metagpt/actions/action.py", line 47, in _aask return await self.llm.aask(prompt, system_msgs) File "/app/metagpt/metagpt/provider/base_gpt_api.py", line 44, in aask rsp = await self.acompletion_text(message, stream=True) File "/app/metagpt/metagpt/provider/openai_api.py", line 32, in wrapper return await f(*args, *kwargs) File "/app/metagpt/metagpt/provider/openai_api.py", line 218, in acompletion_text return await self._achat_completion_stream(messages) File "/app/metagpt/metagpt/provider/openai_api.py", line 151, in _achat_completion_stream response = await openai.ChatCompletion.acreate( File "/usr/local/lib/python3.9/site-packages/openai/api_resources/chat_completion.py", line 45, in acreate return await super().acreate(args, kwargs) File "/usr/local/lib/python3.9/site-packages/openai/api_resources/abstract/engine_apiresource.py", line 217, in acreate response, , api_key = await requestor.arequest( File "/usr/local/lib/python3.9/site-packages/openai/api_requestor.py", line 382, in arequest resp, got_stream = await self._interpret_async_response(result, stream) File "/usr/local/lib/python3.9/site-packages/openai/api_requestor.py", line 726, in _interpret_async_response self._interpret_response_line( File "/usr/local/lib/python3.9/site-packages/openai/api_requestor.py", line 763, in _interpret_response_line raise self.handle_error_response( openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens. However, you requested 4308 tokens (2808 in the messages, 1500 in the completion). Please reduce the length of the messages or completion.
The above exception was the direct cause of the following exception:
Due to the uncertainty of LLM's output, we cannot guarantee a successful outcome with every run. However, what we can ascertain is that the success rate of the GPT-4.0 model is higher than that of the GPT-3.5-turbo.
pydantic.error_wrappers.ValidationError: 5 validation errors for prd Requirement Pool -> 0 value is not a valid tuple (type=type_error.tuple) Requirement Pool -> 1 value is not a valid tuple (type=type_error.tuple) Requirement Pool -> 2 value is not a valid tuple (type=type_error.tuple) Requirement Pool -> 3 value is not a valid tuple (type=type_error.tuple) Anything UNCLEAR field required (type=value_error.missing)
Hello, I also meet this problem when replace GPT4 with local LLM. Have you solved this problem? How to process this error. Thank you!
Hi,
Maybe a stupid question, since the api we are using is
chat models listed in this page https://platform.openai.com/docs/models/overview should be our available models at this moment, right?