fluree / db

Fluree database library
https://fluree.github.io/db/
Other
337 stars 21 forks source link

Content addressed storage #766

Closed zonotope closed 4 months ago

zonotope commented 5 months ago

This patch moves our storage implementations to be more content addressed. There were a few places where code outside the storage implementation produced content hashes that were then passed down and used by the storage implementation, and these hashes often didn't match the data that was actually being stored. This patch relies on the storage implementations themselves to produce content hashes and faithfully stores the data under those hashes that was passed to it. All of the paths and addresses are also generated by the storage implementations and not by any code outside of it. This should further separate the storage concern to be behind the storage abstraction and should help these storage records to be truly interchangeable.

This patch also removes the ".json" suffix from stored files because we can't assume all the files we store will always be json, and it removes address from the storage abstraction because the address because the address will only be reliably known after the data is stored, and nothing outside of the storage abstraction should need to generate an address outside of the context of storing data, so they should rely on the :address field of the map returned by a successful storage operation.

I also found one error in legacy nameservice lookups where an async operation was treated as synchronous. I fixed that issue in https://github.com/fluree/db/commit/95ba0919119dfc538ec7bfec31a794e87ecb8aa4 and https://github.com/fluree/db/commit/ff60dd2e091feeb55753b8b90fd0bb3d9d076134