fix: reduce max page to crawl on config

BuilderIO / gpt-crawler

Crawl a site to generate knowledge files to create your own custom GPT from a URL

https://www.builder.io/blog/custom-gpt

ISC License

18.6k stars 1.97k forks source link

Closed pipech closed 10 months ago

pipech commented 10 months ago

New use will run with current config to test out the repo.

With 1,000 max page to crawl, if there are traffic on this repo there will be unnecessary heavy load on your server.

So I change max page to crawl from 1,000 to 5.

steve8708 commented 10 months ago

thanks @pipech !

github-actions[bot] commented 10 months ago

:tada: This PR is included in version 1.0.0 :tada:

The release is available on:

Your semantic-release bot :package::rocket: