Support for Prefetching Cross-Store References

[tmagrino]

In Fabric's store, we use object groups as an attempt to bundle and prefetch related objects in batches for the worker. However, there is currently no support for prefetching objects pointed to on other stores in a similar fashion.

In the inlined-treaties branch, I wrote up a somewhat unsafe feature for supporting this through the notion of "associate" objects, which get prefetched from other stores when adding the referring object to a worker's cache. There's probably a safer general design worth considering for the master branch of Fabric to address this in general.

[tmagrino]

From discussion with @andrewcmyers over email and slack:

I see various options:

1. the object has a special field that indicates other objects to prefetch. This can be set at object creation time and overridden later. A bit clunky and comes with significant storage overhead but allows the server to do things without unswizzling objects.
2. the object has a method that returns objects to prefetch along with it. This gives maximum flexibility but has more cost at the server.
3. there is a special operation that can be invoked by a transaction to prefetch a set of objects, perhaps incorporating a traversal to some depth as well.
4. A class can mark some of its fields as things to be prefetched. This is the least flexible but it has no storage overhead.

Strategies # 1 and # 2 above could be special cases of # 4.

Tom: I provide a field that's an inlined set of oids that you can add to

For # 1, the class would specify that you should look at the field that # 1 defines to determine what to prefetch.

The advantage of prefetching strategy # 3 is that you don’t ask for things you don’t need. The disadvantage is that you can’t ask for things you don’t know about.

I suspect there are really two different mechanisms at play here, depending on whether you are fetching an object for the first time or fetching an object because you know you have an old version. In the first case, there is not much concern that you are refetching a referenced object you already have. In the second case, you mostly need to know only about referenced objects that have been updated or objects that are newly referenced by the new version of the fetched object.