search

stitionai / devika

Devika is an Agentic AI Software Engineer that can understand high-level human instructions, break them down into steps, research relevant information, and write code to achieve the given objective. Devika aims to be a competitive open-source alternative to Devin by Cognition AI.
MIT License
18.39k stars 2.38k forks source link

Playwright: Protocol Error/ Not getting links with Bing or Google as search engines #368

Closed cardchase closed 6 months ago

cardchase commented 6 months ago

I tried to replicate the demo video and have run into a roadblock.

Logs:

24.04.05 15:56:51: root: INFO   : SOCKET server-message MESSAGE: {'messages': {'from_devika': False, 'message': "implement Conway's game of life in Python using PyGame", 'timestamp': '2024-04-05 15:56:51'}}
24.04.05 15:56:51: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 746}
Model: claude-3-haiku-20240307, Enum: CLAUDE
24.04.05 15:56:56: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 1117}

plan ::  ```
Project Name: Conway's Game of Life

Your Reply to the Human Prompter: Sure, I'd be happy to create a step-by-step plan to implement Conway's Game of Life in Python using PyGame. As an AI software engineer, I'll leverage my knowledge and research to provide a comprehensive plan to tackle this task.

Current Focus: Implement Conway's Game of Life simulation using Python and PyGame.

Plan:
- [ ] Step 1: Research and understand the rules of Conway's Game of Life.
- [ ] Step 2: Set up the PyGame environment and create a basic grid representation for the game.
- [ ] Step 3: Implement the logic to apply the Game of Life rules and update the grid state.
- [ ] Step 4: Render the grid state using PyGame's graphics functions and update the display in real-time.
- [ ] Step 5: Add user interaction features, such as the ability to toggle the state of individual cells.
- [ ] Step 6: Optimize the simulation performance and handle large grid sizes efficiently.
- [ ] Step 7: Add additional features, such as pause/resume functionality, generation counters, or visualization options.
- [ ] Step 8: Test the implementation thoroughly and fix any bugs or issues.
- [ ] Step 9: Document the code and provide instructions for running the simulation.

Summary: This plan outlines the necessary steps to implement Conway's Game of Life using Python and PyGame. It covers the initial research, setting up the development environment, implementing the game logic, rendering the simulation, adding user interactivity, and optimizing the performance. The plan also includes steps for testing and documentation to ensure a complete and functional solution. The key considerations are efficient grid management, handling large grid sizes, and providing a smooth and engaging user experience.

24.04.05 15:56:56: root: INFO   : SOCKET agent-state MESSAGE: [{'internal_monologue': None, 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 745, 'timestamp': '2024-04-05 15:56:51'}]
24.04.05 15:56:56: root: INFO   : SOCKET server-message MESSAGE: {'messages': {'from_devika': True, 'message': "Sure, I'd be happy to create a step-by-step plan to implement Conway's Game of Life in Python using PyGame. As an AI software engineer, I'll leverage my knowledge and research to provide a comprehensive plan to tackle this task.", 'timestamp': '2024-04-05 15:56:56'}}
24.04.05 15:56:56: root: INFO   : SOCKET server-message MESSAGE: {'messages': {'from_devika': True, 'message': '{\n    "1": "Research and understand the rules of Conway\'s Game of Life.",\n    "2": "Set up the PyGame environment and create a basic grid representation for the game.",\n    "3": "Implement the logic to apply the Game of Life rules and update the grid state.",\n    "4": "Render the grid state using PyGame\'s graphics functions and update the display in real-time.",\n    "5": "Add user interaction features, such as the ability to toggle the state of individual cells.",\n    "6": "Optimize the simulation performance and handle large grid sizes efficiently.",\n    "7": "Add additional features, such as pause/resume functionality, generation counters, or visualization options.",\n    "8": "Test the implementation thoroughly and fix any bugs or issues.",\n    "9": "Document the code and provide instructions for running the simulation. "\n}', 'timestamp': '2024-04-05 15:56:56'}}

context_keywords ::  ['pygame', 'simulation', 'conway', 'life', 'using'] 

24.04.05 15:56:58: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 1739}
Model: claude-3-haiku-20240307, Enum: CLAUDE
24.04.05 15:57:00: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 1384}

internal_monologue ::  Hmm, let's see. I've got a great plan to implement Conway's Game of Life using Python and PyGame. First, I need to make sure I understand the rules of the game, then I can start setting up the development environment. This is going to be a fun project to work on! 

24.04.05 15:57:00: root: INFO   : SOCKET agent-state MESSAGE: [{'internal_monologue': None, 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 1313, 'timestamp': '2024-04-05 15:56:51'}, {'internal_monologue': "Hmm, let's see. I've got a great plan to implement Conway's Game of Life using Python and PyGame. First, I need to make sure I understand the rules of the game, then I can start setting up the development environment. This is going to be a fun project to work on!", 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 0, 'timestamp': '2024-04-05 15:57:00'}]
24.04.05 15:57:00: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 1908}
Model: claude-3-haiku-20240307, Enum: CLAUDE
24.04.05 15:57:01: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 1004}

research ::  {'queries': ["Conway's Game of Life rules"], 'ask_user': ''} 

24.04.05 15:57:01: root: INFO   : SOCKET server-message MESSAGE: {'messages': {'from_devika': True, 'message': "I am browsing the web to research the following queries: Conway's Game of Life rules.\n If I need anything, I will make sure to ask you.", 'timestamp': '2024-04-05 15:57:01'}}
24.04.05 15:57:01: root: INFO   : SOCKET agent-state MESSAGE: [{'internal_monologue': None, 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 1313, 'timestamp': '2024-04-05 15:56:51'}, {'internal_monologue': "Hmm, let's see. I've got a great plan to implement Conway's Game of Life using Python and PyGame. First, I need to make sure I understand the rules of the game, then I can start setting up the development environment. This is going to be a fun project to work on!", 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 979, 'timestamp': '2024-04-05 15:57:00'}]
24.04.05 15:57:01: root: INFO   : 
Search Engine :: google
Searching in Google...

Link ::   

Exception in thread Thread-9 (<lambda>):
Traceback (most recent call last):
  File "/home/harshad/miniconda3/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/home/harshad/miniconda3/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/home/harshad/devika/devika.py", line 96, in <lambda>
    thread = Thread(target=lambda: agent.execute(message, project_name))
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/harshad/devika/src/agents/agent.py", line 348, in execute
    search_results = self.search_queries(queries, project_name)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/harshad/devika/src/agents/agent.py", line 111, in search_queries
    browser, raw, data = loop.run_until_complete(self.open_page(project_name, link))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/harshad/miniconda3/lib/python3.12/asyncio/base_events.py", line 684, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/harshad/devika/src/agents/agent.py", line 72, in open_page
    await browser.go_to(pdf_download_url)
  File "/home/harshad/devika/src/browser/browser.py", line 32, in go_to
    await self.page.goto(url, timeout=20000)
  File "/home/harshad/miniconda3/lib/python3.12/site-packages/playwright/async_api/_generated.py", line 8612, in goto
    await self._impl_obj.goto(
  File "/home/harshad/miniconda3/lib/python3.12/site-packages/playwright/_impl/_page.py", line 500, in goto
    return await self._main_frame.goto(**locals_to_params(locals()))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/harshad/miniconda3/lib/python3.12/site-packages/playwright/_impl/_frame.py", line 145, in goto
    await self._channel.send("goto", locals_to_params(locals()))
  File "/home/harshad/miniconda3/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 59, in send
    return await self._connection.wrap_api_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/harshad/miniconda3/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 509, in wrap_api_call
    return await cb()
           ^^^^^^^^^^
  File "/home/harshad/miniconda3/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 97, in inner_send
    result = next(iter(done)).result()
             ^^^^^^^^^^^^^^^^^^^^^^^^^
playwright._impl._errors.Error: Protocol error (Page.navigate): Cannot navigate to invalid URL

What am I missing?

cardchase commented 6 months ago

I am facing the same playwright._impl._errors.Error: Protocol error (Page.navigate): Cannot navigate to invalid URL with local mistral too

ARajgor commented 6 months ago

there are no Links :: that's why it throws the error. let me check what causing that

Ahmet0691 commented 6 months ago

def get_first_link(self): if 'items' in self.query_result: for item in self.query_result['items']: if 'link' in item: return item['link'] return ""

cardchase commented 6 months ago

def get_first_link(self): if 'items' in self.query_result: for item in self.query_result['items']: if 'link' in item: return item['link'] return ""

Where am I supposed to edit this code?

Alwin8 commented 6 months ago

I think that is a error from google api . So use duck duck go or bing instead. The error will be solved

cardchase commented 6 months ago

So I changed the search engine to bing and I get nonetype error. for some reason my workspace is ont getting any links I guess. 

Socket connected ::  {'data': 'frontend connected!'}
24.04.06 22:20:34: root: INFO   : SOCKET socket_response MESSAGE: {'data': 'Server Connected'}
24.04.06 22:20:38: root: INFO   : /api/get-agent-state POST
24.04.06 22:20:38: root: DEBUG  : /api/get-agent-state POST - Response: {"state":{"agent_is_active":true,"browser_session":{"screenshot":null,"url":null},"completed":false,"internal_monologue":"Hmm, let's see. I've got a great plan to implement Conway's Game of Life using Python and PyGame. First, I need to make sure I understand the rules of the game, then I can start setting up the development environment. This is going to be a fun project to work on!","message":null,"step":null,"terminal_session":{"command":null,"output":null,"title":null},"timestamp":"2024-04-05 15:57:00","token_usage":979}}

24.04.06 22:20:51: root: INFO   : /api/create-project POST
24.04.06 22:20:51: root: DEBUG  : /api/create-project POST - Response: {"message":"Project created"}

24.04.06 22:20:51: root: INFO   : /api/get-agent-state POST
24.04.06 22:20:51: root: DEBUG  : /api/get-agent-state POST - Response: {"state":null}

24.04.06 22:20:54: root: INFO   : /api/calculate-tokens POST
24.04.06 22:20:54: root: DEBUG  : /api/calculate-tokens POST - Response: {"token_usage":11}

24.04.06 22:21:01: root: INFO   : SOCKET server-message MESSAGE: {'messages': {'from_devika': False, 'message': "implement Conway's game of life in Python using PyGame", 'timestamp': '2024-04-06 22:21:01'}}
24.04.06 22:21:01: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 746}
Model: claude-3-haiku-20240307, Enum: CLAUDE
24.04.06 22:21:06: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 1017}

plan ::  ```
Project Name: Conway's Game of Life

Your Reply to the Human Prompter: Certainly! I'd be happy to create a step-by-step plan to implement Conway's Game of Life in Python using PyGame. Let me get started on that for you.

Current Focus: Implementing the core logic of Conway's Game of Life and visualizing the simulation using PyGame.

Plan:
- [ ] Step 1: Research and understand the rules of Conway's Game of Life.
- [ ] Step 2: Design the data structure to represent the game grid and the state of each cell.
- [ ] Step 3: Implement the logic to update the state of the cells based on the rules of the game.
- [ ] Step 4: Set up the PyGame environment and create the game window.
- [ ] Step 5: Render the game grid and update the cell states in the PyGame window.
- [ ] Step 6: Implement user interactions, such as the ability to toggle the state of cells.
- [ ] Step 7: Add additional features, such as the ability to pause, resume, or reset the simulation.
- [ ] Step 8: Test the game implementation and fix any issues or bugs.

Summary: This plan outlines the necessary steps to implement Conway's Game of Life in Python using PyGame. The key considerations include understanding the game rules, designing an efficient data structure, and integrating the game logic with the PyGame visualization. Potential challenges may arise in managing the game grid's state updates and ensuring smooth rendering in the PyGame window.

24.04.06 22:21:06: root: INFO   : SOCKET agent-state MESSAGE: [{'internal_monologue': None, 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 695, 'timestamp': '2024-04-06 22:21:01'}]
24.04.06 22:21:06: root: INFO   : SOCKET server-message MESSAGE: {'messages': {'from_devika': True, 'message': "Certainly! I'd be happy to create a step-by-step plan to implement Conway's Game of Life in Python using PyGame. Let me get started on that for you.", 'timestamp': '2024-04-06 22:21:06'}}
24.04.06 22:21:06: root: INFO   : SOCKET server-message MESSAGE: {'messages': {'from_devika': True, 'message': '{\n    "1": "Research and understand the rules of Conway\'s Game of Life.",\n    "2": "Design the data structure to represent the game grid and the state of each cell.",\n    "3": "Implement the logic to update the state of the cells based on the rules of the game.",\n    "4": "Set up the PyGame environment and create the game window.",\n    "5": "Render the game grid and update the cell states in the PyGame window.",\n    "6": "Implement user interactions, such as the ability to toggle the state of cells.",\n    "7": "Add additional features, such as the ability to pause, resume, or reset the simulation.",\n    "8": "Test the game implementation and fix any issues or bugs. "\n}', 'timestamp': '2024-04-06 22:21:06'}}

context_keywords ::  ['pygame', 'conway', 'simulation', 'life', 'core'] 

24.04.06 22:21:08: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 1589}
Model: claude-3-haiku-20240307, Enum: CLAUDE
24.04.06 22:21:10: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 1320}

internal_monologue ::  Okay, let's dive into this step-by-step plan to implement Conway's Game of Life. First, I need to make sure I fully understand the game rules and how to represent the grid and cell states. Then I can start setting up the PyGame environment and bring this simulation to life. I'm excited to see this come together and add some fun user interaction features. Let's get started! 

24.04.06 22:21:10: root: INFO   : SOCKET agent-state MESSAGE: [{'internal_monologue': None, 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 1231, 'timestamp': '2024-04-06 22:21:01'}, {'internal_monologue': "Okay, let's dive into this step-by-step plan to implement Conway's Game of Life. First, I need to make sure I fully understand the game rules and how to represent the grid and cell states. Then I can start setting up the PyGame environment and bring this simulation to life. I'm excited to see this come together and add some fun user interaction features. Let's get started!", 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 0, 'timestamp': '2024-04-06 22:21:10'}]
24.04.06 22:21:10: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 1808}
Model: claude-3-haiku-20240307, Enum: CLAUDE
24.04.06 22:21:13: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 1396}
Invalid response from the model, trying again...
24.04.06 22:21:13: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 2958}
Model: claude-3-haiku-20240307, Enum: CLAUDE
24.04.06 22:21:17: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 2754}
Invalid response from the model, trying again...
24.04.06 22:21:17: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 4212}
Model: claude-3-haiku-20240307, Enum: CLAUDE
24.04.06 22:21:18: root: INFO   : SOCKET tokens MESSAGE: {'token_usage': 3358}

research ::  {'queries': ["Conway's Game of Life rules"], 'ask_user': ''} 

24.04.06 22:21:18: root: INFO   : SOCKET server-message MESSAGE: {'messages': {'from_devika': True, 'message': "I am browsing the web to research the following queries: Conway's Game of Life rules.\n If I need anything, I will make sure to ask you.", 'timestamp': '2024-04-06 22:21:18'}}
24.04.06 22:21:18: root: INFO   : SOCKET agent-state MESSAGE: [{'internal_monologue': None, 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 1231, 'timestamp': '2024-04-06 22:21:01'}, {'internal_monologue': "Okay, let's dive into this step-by-step plan to implement Conway's Game of Life. First, I need to make sure I fully understand the game rules and how to represent the grid and cell states. Then I can start setting up the PyGame environment and bring this simulation to life. I'm excited to see this come together and add some fun user interaction features. Let's get started!", 'browser_session': {'url': None, 'screenshot': None}, 'terminal_session': {'command': None, 'output': None, 'title': None}, 'step': None, 'message': None, 'completed': False, 'agent_is_active': True, 'token_usage': 3333, 'timestamp': '2024-04-06 22:21:10'}]
24.04.06 22:21:18: root: INFO   : 
Search Engine :: bing
Exception in thread Thread-3 (<lambda>):
Traceback (most recent call last):
  File "/home/harshad/miniconda3/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/home/harshad/miniconda3/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/home/harshad/devika/devika.py", line 96, in <lambda>
    thread = Thread(target=lambda: agent.execute(message, project_name))
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/harshad/devika/src/agents/agent.py", line 348, in execute
    search_results = self.search_queries(queries, project_name)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/harshad/devika/src/agents/agent.py", line 108, in search_queries
    link = web_search.get_first_link()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/harshad/devika/src/browser/search.py", line 32, in get_first_link
    return self.query_result["webPages"]["value"][0]["url"]
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
TypeError: 'NoneType' object is not subscriptable

But duck duck go was a progress. I read some code in the terminal that I pasted in copilot and I got an app type. I am not a pro here but I got blank screen. I suppose bing usually resolves these bugs for me but then I found the main.py file and I ran it, the same. Did not get an app as the demo but initiated pygame lol. Will post a new bug on it but the project looks incredibly promising. Imagine in my business give a requirement "i want to invoice kraft for the current month. give me a pdf template to look at and save it in word format as well at 'this location'" Is gold in the line of work I work in. I have unhappy people doing this job lol.

ngupta181 commented 6 months ago

Hey @cardchase,

It seems like you're missing GOOGLE_SEARCH_ENGINE_ID = "" in the Config.toml file. You can refer to this document on how to get your Google Custom Search API Key and GOOGLE_SEARCH_ENGINE_ID: (https://developers.google.com/custom-search/v1/introduction).

I encountered the same issue before and managed to fix it.

cardchase commented 6 months ago

@ngupta181 I tried the process. I got the API key however I could not find the seach engine ID anywhere nor the names that were displayed worked. I am gettinng the "no valid urls found' error at the terminal end. how do I get my search engine ID?

ARajgor commented 6 months ago

solved it. fetch the latest changes.