Local del of data for storage cleaning

[mjp0]

Is there some way to remove the data behind key values locally without replicating changes? A bit like TTL for storing the data itself or "selective replication".

I'm working on a project where a large part of the servers used for processing have a very constrained storage space. I was thinking about using hyperdb to transfer data but without some sort of way to clear the processed data, those servers are going to fill up quickly.

One way would be to do something like /unprocessed and /processed, and then watch /unprocessed in the small servers. After processing server would move data to /processed which is watched in the backup server. However the problem here is that if the server writes to /processed it will store the data no matter whether it's replicating that particular root key.

Any ideas?

[mjp0]

I just checked hypercore a bit and saw storageCacheSize setting. Maybe this can be used here?

[pfrazee]

@0fork storageCacheSize controls memory usage. If you want to delete locally-stored data, use the hypercore's clear() method.

[mjp0]

@pfrazee thanks!