djdv / go-filesystem-utils

ISC License
10 stars 2 forks source link

Various fixes for IPFS directory handling #38

Closed djdv closed 1 year ago

djdv commented 1 year ago

Fixes: https://github.com/djdv/go-filesystem-utils/issues/29 This includes a number of drive-by fixes in other areas as well, but the main focus is on 99b7af8ca1bbf2b7a416300a555ac9651552faaa and later. (Not worth extracting the others and submitting them separately right now.)

Sharded directory nodes should resolve correctly with these patches, and some other potential problems that would result in bad behaviour were found and fixed (bad return values and deadlocks when receiving errors during directory operations).


The implementation of the resolver originally used the core API directly which implied all our operations would be making requests to the node even if we already had some data cached locally. (The commit also needed a fix). Since we already implemented caching (for the same kind of data the resolver interface also requires), it made sense to utilize that instead of always requesting new data from a (potentially remote and/or slow) node.

This implementation is somewhat ugly, but it seems to work so I'm merging it now. I think a proper solution will be to tackle https://github.com/djdv/go-filesystem-utils/issues/30 which should obviate most of this code and have us going back to making requests to the node via the core API, but caching its responses for the various data we'll need (mitigating any repeated request costs).

This puts all of the responsibility on to the node and its implementation, which has advantages and disadvantages. We'd need to depend on and communicate with it more, which might impact performance, but our code shouldn't have to adapt or handle many edge cases (like the one this pr fixes) since it's the node's responsibility to handle our high level requests correctly.