Store netDB and peerProfiles in a key-value storage

[l-n-s]

To address the issue of high disk usage by NetDb and PeerProfiles, which is especially bad for devices with low flash and/or RAM (when using tmpfs), like cheap routers, Raspberry Pi and Anrdoid SDCard.

[orignal]

I see too different problems:

1. Excessive disk space usage
2. Excessive I/O operations

While 1. could be implemented with any approach using a single big file instead bunch of small, 2. is a subject for discussions and suggestion. Using any additional libraries wouldn't solve 2. automatically because amount of data remains the same.

[hakunamtu]

As I can see, some sources of (write)I/O operations in NetDb are storing info on updated routers and storing profiles for unreachable ones. One of solutions is to store this information in some sort of buffer, from which records are flushed on a timeout or on repletion. I don't know, how often routers become reachable in a short time after being unreachable, maybe this buffer could be a queryable part of a new storage.

[hakunamtu]

Another way to reduce disk I/O might be making netDb and peerPorfiles disk synchronisation interval configurable. It will allow users to make disk writes/RAM consumption tradeoffs themselves, interval value 0 might stand for no sync at all (don't know what would be better to do with peerProfiles in this case: maybe, don't keep them at all, maybe keep them in size-restricted LRU cache) . If it's possible to add external dependency, I suggest TDB: it's small (43k) and widely used (because part of samba). If not, I will be glad to write custom storage (probably, based on HAMT, since all keys are equal in size), though this effort will likely end up in reimplementation of most of already existing solutions' features.

[orignal]

Go ahead and implement these config params

[hakunamtu]

Synchronization interval option: #1285 What about TDB storage backend?

[orignal]

if you wish try to implement a flat file (tar like) for entire storage. Meaning you store all files sequentially in one shot. But without additional dependencies, please.