Open ecton opened 2 years ago
RE: The mapping function should be able to decide not to update the record if nothing relevant has changed
-> For the majority of cases, I think maybe defining a custom hash function might suffice for checking whether invalidation is necessary. This way, you don't necessarily have to keep the prev value for checking at all, you can just call the custom hashing value and compare that to the hash stored against the document for that view.
Perhaps there are multiple strategies for invalidation, which could have interesting implications, like:
trait InvalidationStrategy<DocumentContent> {
/// To be stored when cache invalidated
///
/// hmm: Is there an interest in storing a small buffer of say 3 item HashMap<CacheValue, ViewValue> so
/// you can use previous computations?
type CacheValue: Serialize + Deserialize;
/// Lazy? – upon pull of view value?
fn map_value(content: CollectionDocument<DocumentContent>) -> CacheValue;
/// Returning true indicates that stored View value is good enough?
fn is_valid(previous: CacheValue, next: CacheValue) -> bool;
}
I agree that what I wrote it a little too simplistic -- purposefully so. I didn't want to narrow the implementation to a specific approach yet. I think it's hard to know what the ergonomics will be without playing with the feature a little bit.
I agree that a hash-based implementation should be possible. But, other implementations that don't rely on hashes should be possible too -- that's why I didn't want to call out a specific approach to how to solve allowing a mapping function to determine if there are changes or not. The requirement that it must be possible is documented, but the exact approach is still to be determined.
@colelawrence suggested a use case for a lazily-evaluated collection, which act sort of like a View but with different re-caching behaviors.
Imagine this common use case: you have uncompressed data stored in the database and you want to compress it before sending it over the network. If you were to create a view that stores the compressed representation as values, you have two problems:
For data where it isn't changing often and all data is expected to be requested, the View system is perfectly find for smaller chunks of data.
This request is to consider creating a lazily evaluated collection:
While one benefit of this collection is that it can be "individually" lazy, I think the fact a mapping function can optionally decide not to update the stored contents makes it a candidate for building Views atop. I see no reason it can't be made to be performant.