search

etix / mirrorbits

Mirrorbits is a geographical download redirector written in Go for distributing files efficiently across a set of mirrors.
MIT License
497 stars 90 forks source link

redis alternatives #78

Closed PalinuroSec closed 6 years ago

PalinuroSec commented 6 years ago

Hello, it's me again.

We tried to configure a little network of distributed redirectors with mirrorbits, but the redis server requires too much ram to hold the whole db for all the mirrors, and we can't perform a ram upgrade on such nodes.

Is it possible to use alternative non-in-memory databases? (what about mongo or postgres)

etix commented 6 years ago

Redis only holds the metadata of the files and mirrors in memory. At VideoLAN, with more than 14K files and 100 mirrors we're under 1GB of RAM allocated.

Also we have no plans of supporting any other database right now.

PalinuroSec commented 6 years ago

ok thanks, i'll give it another try

we currently have 262709 files over 50+ mirrors