search

mendableai / firecrawl

🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.
https://firecrawl.dev
GNU Affero General Public License v3.0
17.39k stars 1.26k forks source link

Are all the links extracted by extractLinksFromHTML the same? #769

Closed zhk0603 closed 1 week ago

zhk0603 commented 1 week ago

https://github.com/mendableai/firecrawl/blob/5ab52854b9d49983f1e2ce10e4b4fb8585c8de42/apps/api/src/scraper/WebScraper/crawler.ts#L372C86-L372C93

通过 extractLinksFromHTML 提取的连接 所有的 content 一样？

nickscamara commented 1 week ago

This is the old crawler implementation. We have moved away from that quite a few months ago. Most of crawler.ts isn't used anymore