cldellow / datasette-scraper

Add website scraping abilities to Datasette
Apache License 2.0
60 stars 1 forks source link

support scheduled crawls #10

Open cldellow opened 1 year ago

cldellow commented 1 year ago

This will require schema changes plus something cron-esque.

simonw commented 1 year ago

For a while I've been thinking that Datasette could benefit from its own cron-like mechanism - maybe as a datasette-cron plugin.

I've not got any further than thinking it might be useful though!

This library looked like it might be a good fit: https://schedule.readthedocs.io/en/stable/background-execution.html