pwliwanow
commented
4 years ago @ngbinh thanks for the kind words and I am glad you like the library!
It's a good suggestion, and the idea fits to other abstractions that are already provided. Here is what I think implementation should support:
1) Integration with schema module (in the future this module will probably become main part of the library).
1) I think that for chunked namespaces, types for all keys should end with SomePrefix :: Int :: HNil (Int would represent which part of a chunk given value is). Then a question arises how will it support schema evolution for keys? One solution would be to have SomePrefix wrapped in another tuple. But then of course compatibility with regular Namespace would be lost. Probably that wouldn't be an issue for users, so I think that pushing that complexity to the library is an acceptable solution.
1) Maybe there should be created additional row (last or first?) that could hold metadata associated with the large value (e.g. image type in case someone wants to store images in that namespace)?
1) Integration with akka-streams module - can be useful for people who at first want to store chunked values (e.g. images) in FoundationDB, but then want to migrate to e.g. S3. It's up to the person implementing whether new Source should be created for efficient streaming data from FoundationDB (without unnecessary transaction restarts)
And probably more things should be taken care of, that will become clear during implementation :)
Unfortunately, I am not gonna have a lot of free time during the next 2-3 months, so if anyone is interested in picking that up, that would be great!
Thanks for this awesome library. I am wondering if the technique described here https://apple.github.io/foundationdb/blob.html can be implemented in foundationdb4s to handle large value size?