search

BuilderIO / gpt-crawler

Crawl a site to generate knowledge files to create your own custom GPT from a URL
https://www.builder.io/blog/custom-gpt
ISC License
18.14k stars 1.88k forks source link

Extracting Articles from Sequential Pages of a Website #131

Open nicofierrov opened 5 months ago

nicofierrov commented 5 months ago

I have a URL with 10 articles and afterwards it is necessary to go to the next page of the URL.

For example:

Is there any way to extract the 10 articles from the first page, the 10 articles from the second page, and so on?