search

Skyvern-AI / skyvern

Automate browser-based workflows with LLMs and Computer Vision
https://www.skyvern.com
GNU Affero General Public License v3.0
10.78k stars 739 forks source link

Scraping Issue #1127

Closed devinat1 closed 1 week ago

devinat1 commented 3 weeks ago

I keep getting the following issue when running on the latest Skyvern:

Traceback (most recent call last):                                                                                                                                                                                                                         
  File "/home/bond/Desktop/agent-collector/agents/open-source/skyvern/skyvern/webeye/scraper/scraper.py", line 254, in scrape_website                                                                                                                      
    return await scrape_web_unsafe(browser_state, url, cleanup_element_tree, scrape_exclude)                                                                                                                                                               
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                               
  File "/home/bond/Desktop/agent-collector/agents/open-source/skyvern/skyvern/webeye/scraper/scraper.py", line 359, in scrape_web_unsafe                                                                                                                   
    raise Exception("No elements found on the page")                                                                         
Exception: No elements found on the page                      
2024-11-04T19:52:08.642444Z [error    ] Unexpected exception in agent_step, marking step as failed step_id=stp_322409123986775562 step_order=3 step_retry=1 task_id=tsk_322408398137302372
Traceback (most recent call last):                            
  File "/home/bond/Desktop/agent-collector/agents/open-source/skyvern/skyvern/webeye/scraper/scraper.py", line 254, in scrape_website
    return await scrape_web_unsafe(browser_state, url, cleanup_element_tree, scrape_exclude)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bond/Desktop/agent-collector/agents/open-source/skyvern/skyvern/webeye/scraper/scraper.py", line 359, in scrape_web_unsafe
    raise Exception("No elements found on the page")                                                                         
Exception: No elements found on the page                      

During handling of the above exception, another exception occurred:                                                          

Traceback (most recent call last):                            
  File "/home/bond/Desktop/agent-collector/agents/open-source/skyvern/skyvern/forge/agent.py", line 594, in agent_step
    ) = await self._build_and_record_step_prompt(                                                                            
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                            
  File "/home/bond/Desktop/agent-collector/agents/open-source/skyvern/skyvern/forge/agent.py", line 1114, in _build_and_record_step_prompt
    scraped_page = await self._scrape_with_type(                                                                             
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                             
  File "/home/bond/Desktop/agent-collector/agents/open-source/skyvern/skyvern/forge/agent.py", line 1089, in _scrape_with_type
    return await scrape_website(                              
           ^^^^^^^^^^^^^^^^^^^^^                              
  File "/home/bond/Desktop/agent-collector/agents/open-source/skyvern/skyvern/webeye/scraper/scraper.py", line 267, in scrape_website
    raise Exception("Scraping failed.")                       
Exception: Scraping failed.

This occurs on most sites I run Skyvern on. I am using release v0.1.37 via the contributors setup on Ubuntu 24.

suchintan commented 2 weeks ago

Hmm.. This isn't good

suchintan commented 2 weeks ago

Are you able to try it with the latest version? Are you still having this issue?

devinat1 commented 1 week ago

Unable to reproduce in the latest version