djdv / go-filesystem-utils

ISC License
10 stars 2 forks source link

IPFS: cache stores more data than is necessary #30

Open djdv opened 1 year ago

djdv commented 1 year ago

_Extracted from: https://github.com/djdv/go-filesystem-utils/pull/27#discussion_r1198901710_

Currently we fetch and cache IPLD nodes so that we can reduce the amount of requests made to the IPFS node. We use this data primarily for path traversal (to avoid calling core.Resolve{Path,Node}), but store the entire node, which can contain block data that we currently don't use/need.

At least for IPFS, we should be able to take advantage of the immutability of paths. So we could use the Go fs.FS path as a cache key instead of a cid, and instead store the cid (and whatever else we require) in that record. We can probably do the same thing for IPNS as long as we continue using some cache eviction condition.

Related: https://github.com/djdv/go-filesystem-utils/issues/29 We will depend on path resolution being correct before we can rely on caching the results of a path traversal.