Scraping job data

[Thaslim42]

while i used firecrawl to scrape data from a job site it only scraped data from the initial page. but the actual data is present inside the job title link i wanted to extract that data too how can i achievev it? ...here is a sample screenshot of the page

[nickscamara]

Hey @Thaslim42 could you try running with allowBackwardLinks option? This enables the crawler to navigate from a specific URL to previously linked pages or pages that are not children of the one that you started the crawl.

[Thaslim42]

it still dont worked..any other options?

[nickscamara]

@Thaslim42

ccing @tomkosm

[tomkosm]

@Thaslim42 are you using scrape or crawl? You should use crawl for this, please share all of the options you are using, the url and the result you are getting. Also are you running self host or using the api?

[Thaslim42]

i used firecrawl playground to crawl this url and it only scraped links of unwanted contents like newsletter gallary etc..but the content inside the job title link didnt scarped...here is a ss of params i provided