search

BuilderIO / gpt-crawler

Crawl a site to generate knowledge files to create your own custom GPT from a URL
https://www.builder.io/blog/custom-gpt
ISC License
18.15k stars 1.88k forks source link

[FR] System to pause and resume the task later #79

Open Snowzer91 opened 7 months ago

Snowzer91 commented 7 months ago

Is it possible to add a system for pausing and resuming the task later?

jaroslavmraz commented 1 month ago

Hi,

solution is implemented. You need to normally stop crawler and then resume using CRAWLEE_PURGE_ON_START=0 npm start