Do not scrap from start if a new instance is setup with an existing DB

iPromKnight / zilean

88 stars 7 forks source link

Do not scrap from start if a new instance is setup with an existing DB #16

Closed ragmehos closed 1 month ago

ragmehos commented 2 months ago

It looks like there is local state that is stored where a new instance of zilean on a different system starts scraping from the beginning but does not take the state that's in the database even when the database is shared?

Is it possible to see what is in the database and scrape only the new content of the DMM lists? This will allow us to bring up a new instance with an existing database without scraping from scratch again?

iPromKnight commented 1 month ago

This is handled now - as the meta is stored in the db along with parsed pages in the latest version :)