search

rawler / bithorde

BitHorde F2F Content Distribution System
http://www.bithorde.org
Other
18 stars 6 forks source link

Cached AssetStore-index #31

Open rawler opened 11 years ago

rawler commented 11 years ago

The contents of the assetstore should preferably be cached in RAM, to avoid excessive disk-I/O.

rawler commented 8 years ago

This is partially solved by b726af748073d274c12c9c73d959f86cf033c980, but we can do better.

Caching is a delicate problem, with lots of tuning. Fortunately, there are those who have already done it.

The long-term plan is to replace all per-hash symlinks with a small SQLite-database. This will reduce inodes and load on the filesystem, come with free RAM-caching, and allow easily storing some extra metadata about the assets, such as completion-levels, and correctly used, thread-safety allowing background scans and updates.