search

mendableai / firecrawl

🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.
https://firecrawl.dev
GNU Affero General Public License v3.0
14.74k stars 1.07k forks source link

[Feat] Run actions like clicking, scrolling or advanced waiting on page before extraction #517

Open calebpeffer opened 1 month ago

calebpeffer commented 1 month ago

I've had a few prospects/customers ask me if we could allow them to run Javascript and actions on the page before we extract the data, such as scrolling the page to access content or clicking on an element.

calebpeffer commented 1 month ago

Yeah this has been +1'd already removing the waitfor tag

calebpeffer commented 3 weeks ago

another +1 @safaricd

calebpeffer commented 3 weeks ago

+1

calebpeffer commented 3 weeks ago

+1

mrruby commented 3 weeks ago

+1

I want to scrap booking eg:

https://www.booking.com/searchresults.pl.html?ss=Warsaw&checkin=2024-10-04&checkout=2024-10-06&group_adults=4&no_rooms=1&group_children=0&order=price&nflt=ht_id%3D201%3Broomfacility%3D38

but I need more data that will load on scroll