Passing the same instance of BasicCompletes between client proxy and server may be problematic

[tjaskula]

How current implementation of ICompletes may fail :

1. When the service calls with(outcome) there may or may not be actions functions registered. If there are any then the service thread will execute the ones that are there.
2. If during 1 the client thread is registering a function and it sees that the outcome is set, it will try to execute functions, but it can't if the service is already executing them.
3. If the service is executing functions and runs out of functions, it will give up and return.
4. When the client has exclusive access to the completes and it is registering a new function, it will execute any earlier registered functions and the one it just registered.
5. The client will continue doing 4 until there are no more functions registered.
6. If 1 is happening and the client has already registered all functions, the service thread will execute all of them.

In fact the clients registration of continuations and server execution may kick in unpredictable times. If we use the same instance (as we curerntly do) the transformations applied by the client, may happen before the transformations applied by the actor which is confusing and complicated to ensure consistent ordering of the transformations to be applied. One of the problems is that this is certainly due to the fact that the same instance of BasicCompletes is passed between the client and the server:

1. The proxy creates the instance of BasicCompletes and returns it to the client
2. The same instance is passed to the server through LocalMessage where the server may start executing actions and set results
3. Before the client has time to register all the continuations.

The goal would be to bind the success/error of the completes that the actor method returns to the completes that is handed back to the client by the proxy. This should place in the consumer function that the proxy also creates and put into the actor’s mailbox.

Points to consider:

• mailbox.Send(new LocalMessage... should not require the instance of BasicCompletes created in the proxy. Just the consumer function handed to the client should be enough.
• Actor may complete with a value and apply transformations before returning to the client
• client should see the results once the server returns (no intermediate results)
• client can still register continuations once he has the server returned the value. It doesn't matter at that point

This may also eliminate the need for the CompletesEventually. Today the issue is that CompletesEventually(); followed by Completes().AndThen() or Completes.Await() etc. results an exception being thrown (and swallowed later) which leads to hanging, too. That it hangs silently is really bad

[tjaskula]

@VaughnVernon Could you check if described problem is accurate enough ?

It's linked with #55

[VaughnVernon]

@tjaskula Seems like what I wrote in Slack.

[tjaskula]

@VaughnVernon the first part yes. To set up the context. Then followed the discussion I had with Alexandros