search

gocolly / redisstorage

Redis based storage backend for Colly
https://godoc.org/github.com/gocolly/redisstorage
Apache License 2.0
35 stars 11 forks source link

Make Visited storage expirable #2

Closed josalmi closed 5 years ago

josalmi commented 6 years ago

Add option to configure expiration time for visited keys. Go defaults time.Duration to zero so this should not be breaking change.

This is an easy way for refetching some pages after given interval as well as to clean up old entires from Redis without removing everything.

Solution commented 5 years ago

Hey guys, is anything wrong with this PR? Or why is not merged yet? @kvch @asciimoo Thx for your time :-)