Store traversal/retrieval

[danielsz]

I was wondering how to go about traversal and/or retrieval of the whole store (or part of it). Or listing all of the keys for inspection purposes.

Something like lookup-fwd-iter, maybe.

Thank you in advance.

[whilo]

As I understand, for this traversal it takes the children of the tree nodes and iterates them: https://github.com/datacrypt-project/hitchhiker-tree/blob/55fcd65f40203b70162bcb2624f931d6b10955a1/src/hitchhiker/tree/messaging.clj#L217

If you want to enumerate the keys in a store, then there are different options. Not all stores implement a key iteration scheme (yet). For the file-store you can use: https://github.com/replikativ/konserve/blob/master/src/konserve/filestore.clj#L29 This effectively has to deserialize all values (in sequence, not at once), so be aware that this is a heavy operation (in general). I have not decided how this should be implemented in general, because the iterators give a view on the store and it is mutable. To avoid inconsistencies one would need to reimplement snapshots.

Currently I think the best thing is to use the underlying store iterator (for its keys) and read the key values out of each value. It is not very performant, but I have barely needed it so far. The best thing is for each application/persistent datastructure to track all referenced values imo. You can then also do a copying GC easily. But I am open to suggestions.

[danielsz]

Thank you, @whilo. I went for the suggested solution (accessing the underlying store), and it works for me.

[awb99]

How do I traverse all keys of say the filesystem store? Is there a femo somewhere how to setup a filesystem store so one has access to all the functions of this namespace?