unclecode / crawl4ai

🔥🕷️ Crawl4AI: Open-source LLM Friendly Web Crawler & Scrapper
Apache License 2.0
10.62k stars 745 forks source link

crawler_strategy.set_hook() is not working #117

Closed studio-anurag closed 21 hours ago

studio-anurag commented 1 day ago

@unclecode I see that setting hook is not working as expected. I am setting a delay using code below :

def delay(driver):
        print("Delaying for 5 seconds...")
        time.sleep(5)
        print("Resuming...")

crawler_strategy = LocalSeleniumCrawlerStrategy(verbose=True)
crawler_strategy.set_hook('after_get_url', delay)
print("Hook Created and attached to crawler strategy")
crawler = WebCrawler(verbose = True, crawler_strategy=crawler_strategy)
print("Hook Set to crawler")

Neither print statements are working nor the delay function is executed. The web page is being crawled without the given delay. Any idea?

studio-anurag commented 21 hours ago

Installing new version, it works