Website Scraping Limit is 15 pages?

[voidxd]

Hello,

Is there a way to increase the amount of scraped pages, or give it certain pages we want scraped?

Thank you!

[gharbat]

You can update the value here:

backend-server/app/Http/Listeners/StartRecursiveCrawler.php:48

for something like:

$this->crawl($url, $crawledUrls, 100, $rootUrl, $chatbotId, $dataSource->getId());