[Self host] Locally running Firecrawl cannot return links using /map.

[Z-ZHHH]

I have downloaded the latest repository and run Firecrawl locally, but the new feature /map does not work properly. For instance, for the following two sites:

curl -X POST http://localhost:3002/v1/map     
-H 'Content-Type: application/json'    
-d '{
      "url": "https://dig.watch/"
}'
{"success":true,"links":["https://dig.watch"]}

curl -X POST http://localhost:3002/v1/map     
-H 'Content-Type: application/json'    
-d '{
      "url": "https://www.51cto.com/ai"
}'
{"success":true,"links":["https://www.51cto.com/ai"]}

However, when the URL is https://firecrawl.dev/, it returns some links.

curl -X POST http://localhost:3002/v1/map     
-H 'Content-Type: application/json'    
-d '{
      "url": "https://firecrawl.dev"
}'
{"success":true,"links":["https://firecrawl.dev","https://www.firecrawl.dev/pricing","https://www.firecrawl.dev/blog","https://www.firecrawl.dev/playground","https://www.firecrawl.dev/smart-crawl","https://www.firecrawl.dev/blog/building-knowledge-graphs-from-web-data-camelai-firecrawl","https://www.firecrawl.dev/blog/cloudflare-error-1015-how-to-solve-it","https://www.firecrawl.dev/blog/contradiction-agent","https://www.firecrawl.dev/blog/how-gamma-supercharges-onboarding-with-firecrawl","https://www.firecrawl.dev/blog/data-extraction-using-llms","https://www.firecrawl.dev/blog/firecrawl-x-dify-ai-add-web-data-to-your-rag-applications","https://www.firecrawl.dev/blog/how-to-easily-install-requests-with-pip-and-python","https://www.firecrawl.dev/blog/how-to-quickly-install-beautifulsoup-with-python","https://www.firecrawl.dev/blog/introducing-fire-engine-for-firecrawl","https://www.firecrawl.dev/blog/firecrawl-july-2024-updates","https://www.firecrawl.dev/blog/launch-week-i-day-3-introducing-map-endpoint","https://www.firecrawl.dev/blog/launch-week-i-day-2-doubled-rate-limits","https://www.firecrawl.dev/blog/launch-week-i-day-1-introducing-teams","https://www.firecrawl.dev/blog/lead-gen-business-insights-make-firecrawl","https://www.firecrawl.dev/blog/chat-with-website","https://www.firecrawl.dev/blog/scrape-analyze-airbnb-data-with-e2b","https://www.firecrawl.dev/blog/firecrawl-june-2024-updates","https://www.firecrawl.dev/blog/using-structured-output-and-json-strict-mode-openai","https://www.firecrawl.dev/blog/using-prompt-caching-with-anthropic","https://www.firecrawl.dev/blog/your-ip-has-been-temporarily-blocked-or-banned"]}

[nickscamara]

Hey @Z-ZHHH , /map is still in alpha. We will be pushing full availability for self hosting it in the next following weeks.

[govindup63]

wait what exactly i have to do in this issue? and also can you assign me this issue

[calebpeffer]

@govindup63 this was a test. Deleted the bounty lmk if there are any other issues on the algora side

[govindup63]

yeah, it's fine now. also, I wanted to learn more about this project so that I can contribute significantly, not just bounty hunting all the time. if you could help me out that would be great

[calebpeffer]

yeah, it's fine now. also, I wanted to learn more about this project so that I can contribute significantly, not just bounty hunting all the time. if you could help me out that would be great

Sure, we have a discord if you'd like to join the community!

We'll be adding more bounties soon

[govindup63]

I'll look into it. thanks btw