unclecode / crawl4ai

🔥🕷️ Crawl4AI: Open-source LLM Friendly Web Crawler & Scrapper
Apache License 2.0
16.61k stars 1.23k forks source link

how use wait_for #207

Closed wg20170107 closed 3 weeks ago

wg20170107 commented 1 month ago

wait_for=".crypto-row:nth-child(20)", # Wait for 20 rows to load what's format of this param

Udbhav8 commented 4 weeks ago

https://crawl4ai.com/mkdocs/basic/page-interaction/ its specified here

unclecode commented 3 weeks ago

@wg20170107 V8 allows you to pass either a CSS selector or JavaScript that returns true or false. During crawling, if a CSS selector is passed, it waits for the element to be visible. If JavaScript is passed, it runs the script until it returns true before continuing. The following is a sample code using CSS. Note the css: prefix, which can also be js: for JavaScript.

result = await crawler.arun(
    url="https://example.com",
    wait_for="css:.dynamic-content"  # Wait for element with class 'dynamic-content'
)