[CLOSED] "Unload" objects in a caller frame only loaded to satisfy callee frame

[aionbot]

Issue created by jeff-aion (on Tuesday Oct 02, 2018 at 18:00 GMT)

In our re-entrant persistence logic, every object in a callee frame must either be newly allocated within that frame or must be present in the caller frame, as well. This means that a callee attempting to access an object not loaded in the caller must first cause the caller to perform the load.

While this provides the correct behaviour, it also has a problem for the billing system: the callee pays for its access but who pays for the caller to now access this object?

This problem means that the most easily defined solution is to force the caller to unload any objects it didn't load for itself, when a callee returns to it. While this seems like a waste, it makes the billing consistent and only harms a very rare corner case (re-entrant calls are incredibly uncommon).

Note that, if this is a write, it also need to write-back into the transaction of the caller so that it can load the updated data, if a later load attempts to load the object. We already write-back updates into the objects, after all.

[aionbot]

Comment by jeff-aion (on Tuesday Oct 02, 2018 at 18:23 GMT)

The first step in this change requires that the use of CopyingDeserializer and SerializedInstanceStub.REENTRANT_CALLEE_INSTANCE_TOKEN be rationalized, in ReentrantGraphProcessor.

The rationale being that if the IDeserializer is just behaviour and no per-instance state, then we can trivially re-install it to "unload" an object. As it stands, these are currently backward (CopyingDeserializer contains all the per-instance state).

This shouldn't be that difficult since all current uses of SerializedInstanceStub.REENTRANT_CALLEE_INSTANCE_TOKEN are just in verification logic, but no decision-making (these verifications can just be changed to instanceof checks against the type).

[aionbot]

Comment by jeff-aion (on Wednesday Oct 03, 2018 at 14:03 GMT)

This is more complicated than I originally thought since there is no way to "write-back into the transaction of the caller" given that these transactions are not the root of the data used in the parent call, in the reentrant case. All the transactional semantics must be captured by the relationship between the reentrant frames.

This means that we need to use a somewhat special IDeserializer instance and add a new method for handling this situation. Specifically, we need to leave the instance variables of the unloaded instance in their state written-back from the callee (and leave any instance stubs created in the instance stub map for this frame - doing otherwise would have been incredibly complicated) and install an IDeserializer which can handle a special "already preloaded" kind of call, which we can put in front of any lazyLoad() calls would otherwise be issuing (since we don't want to bother with the object's deserialization logic in this case.

[aionbot]

Comment by jeff-aion (on Wednesday Oct 03, 2018 at 14:12 GMT)

Correction: Since the lazyLoad() can be called from even the user code, we can't guard it with some call to the IDeserializer. However, we only have one actual implementation of lazyLoad() so we can add this special call-out guarding the loading path in that implementation.

[aionbot]

Comment by jeff-aion (on Wednesday Oct 03, 2018 at 14:43 GMT)

The main thing to keep in mind is that we need a well-defined way of handling all of these actions within the reentrant case:

• load (from parent)
• store (to parent)
• commit (from child)
• revert (from child)

We don't yet make a distinction between read and write operations against an object (#268 will introduce this), so we currently only need to consider loaded for child or not.

I was initially trying to handle all of this complexity within the callee but some/all of it actually needs to be handled in the caller since it needs to know to write-back any changed, yet unloaded, objects associated with its frame when it commits.

[aionbot]

Comment by jeff-aion (on Thursday Oct 04, 2018 at 15:56 GMT)

Thinking further about this, it actually seems like the caller might be able to handle all of this as a consequence of however it will decide to do billing on load, or not.

In order to determine how to handle billing decisions, the caller needs to be given a piece of state to describe whether it is the frame currently executing (top of the stack) or whether something is on top of it. If it is on top when a load happens, it can populate the object, bill for the load, and add the object to the list to scan on commit (handles the "island of change" problem). If it is not, it should populate the object, merely record the billing it would have done, add it to a list which may require a "free write-back" on commit, and have a simpler IDeserializer installed (which would require that Object clear the IDeserializer before the call, instead of after, in lazyLoad()).

This simpler IDeserializer exists just for the case where this "pre-loading" has been done. If it is called to load while the owning frame is not on the top, it does nothing and merely installs the IDeserializer back into the loaded instance (this could happen if successive reentrant calls access the same object which the parent frame hasn't loaded). If it is on top, the recorded billing intention is applied, the object is removed from the "free write-back" set, is added to the list scan on commit, and it is left loaded (without an IDeserializer instance).