search

sergiotapia / magnetissimo

Web application that indexes all popular torrent sites, and saves it to the local database.
MIT License
3k stars 190 forks source link

In memory kv for magnet lookup #67

Closed christian-fei closed 6 years ago

christian-fei commented 7 years ago

Hello Sergio,

first of all: thanks! This was a lot of fun, looking at your code, the idea etc.

This PR adds a very simple cache to each crawler, to prevent aggressive/unneeded crawling.

tchoutri commented 6 years ago

Hi! Thanks for your PR!
What are the concrete benefits of having such a cache? Do you have some number / graphs?

ThatLurker commented 6 years ago

This PR is not needed anymore as we moved to using RSS mainly (and api's). So i personally don't see any reason on having a cache for RSS (+ api) lookups.

tchoutri commented 6 years ago

Okay, thanks