microsoft / service-fabric-services-and-actors-dotnet

Reliable Services and Reliable Actors are Service Fabric application frameworks for building highly-scalable distributed cloud applications.
Other
269 stars 115 forks source link

Support Actor self-deletion. #230

Open olitomlinson opened 4 years ago

olitomlinson commented 4 years ago

Is your feature request related to a problem? Please describe.

I'm currently designing a multi-tenant high-throughput system that will be creating approximately 50-150 million short-lived Actors per day depending on customer demand.

An Actor will perform its duty and come to a conclusion, publish its results to EventGrid, and then become obsolete. The Actor does not have a natural owner or supervisor that is in control of its lifetime.

In order to perform this deletion, (because there is no owner....) I must delegate the responsibility of the deletion to an external process, which can then invoke the Actor delete method.

This deletion process could be hosted as a simple and straight-forward Stateful Service, but this is one more service that I need to own and operate, which exists purely as a side-effect of the technology.

In order to keep the services in the domain focused on business value, I would prefer not to have this additional responsibility and cognitive load.

Describe the solution you'd like

I would like an Actor to be able to delete itself.

The Deletion API could be borrow stylistically from the IRemindable interface. Something like :

var _registration = RegisterDeletion(TimeSpan.FromMinutes(15)); //schedule the Actor for deletion in 15 minutes time.
var _registration = RegisterDeletion(0); //schedule the Actor for immediate deletion.
UnregisterDeletion(_registration); //Things have changed, I no longer want to delete the Actor

Describe alternatives you've considered

As described above, I would have to create another service that has the responsibility of performing the deletion. This could be done internally in the cluster by introducing a new Stateful Service to own the process, but would utilise more precious cluster resources, such as :

Additional context

What would happen when deletion is not honoured successfully and internal retries have been exhausted? There is no supervisor or owner to notify. Would it raise a Health Event? In the event of failure, something would need to be logged somewhere as there could potentially be orphaned Actor metadata sitting around in memory forever which must be cleaned-up somehow.

AlkisFortuneFish commented 4 years ago

One workaround until (and if) this is supported is to have the actor fire off deletion with a Task.Run(), without awaiting it and immediately returning.

BuddhaBuddy1 commented 3 years ago

One workaround until (and if) this is supported is to have the actor fire off deletion with a Task.Run(), without awaiting it and immediately returning.

If there were a problem with the task, it would not be deleted. There needs to be a reliable deletion method.

It seems very strange to me that you can't just mark it for deletion, implement OnDeleteAsync(), and the GC deletes it the next sweep.

AlkisFortuneFish commented 3 years ago

If there were a problem with the task, it would not be deleted. There needs to be a reliable deletion method.

Indeed. In newer code I achieve that at the service level, an actor registers itself for deletion by calling code in the actor service, which persists the request and then deletes it afterwards.

It seems very strange to me that you can't just mark it for deletion, implement OnDeleteAsync(), and the GC deletes it the next sweep.

Totally agreed.

BuddhaBuddy1 commented 3 years ago

I figured I would have to do something like that. It is just silly that I have to create a new Stateful Service just to have them be able to reliably delete themselves (I'm invoking them from a Stateless Service).

AlkisFortuneFish commented 3 years ago

You can do it directly on the ActorService itself, with RunAsync periodically running the deletions, although the lack of direct access to the underlying state provider means that you have to persist state via IActorStateProvider and a sacrificial actorId, which feels quite hacky. Unless your actors don't use persisted state of course. All in all, I haven't found any truly nice solutions to this.

olitomlinson commented 3 years ago

Stateful Services programming models getting no love for a few years now. Very disappointing.