ing-bank / baker

Orchestrate microservice-based process flows
https://ing-bank.github.io/baker/
MIT License
321 stars 84 forks source link

Add the possibility to prevent concurrent processing of a SensoryEvent #469

Open dfollen-ing opened 4 years ago

dfollen-ing commented 4 years ago

Hi

We have a case where a part of the recipe is repeatable. The same sensory event which has no firing limit (withSensoryEventNoFiringLimit(RepeatableSensoryEvent.class)) can be processed several times. This case happens for instance when you have gather consent from different actors and they can provide their consent concurrently. The issue we face right now is that the context in which the sensory event is processed is unique for the recipe, meaning that any concurrent sensory event will impact the events and ingredients context. Is it possible to add a functionality to mark a sensory event as non concurrent, (for instance withSensoryEventNoFiringLimitNonConcurrent(RepeatableSensoryEvent.class)). When the sensory event is received a (potential distributed) lock is being acquired. If the lock has already been acquired then the sensory event cannot be processed until the lock is released.

This could be implemented by creating a technical "lock token" (invisible to the recipe) that is needed with the sensory event. The token is then consumed when the sensory event is processed and re-generated when the recipe has finished processing the sensory event. The next sensory event will only be processed when the "lock token" has been re-generated. This "lock token" would be generated at the creation of the recipe.

Tim-Linschoten commented 4 years ago

Hello David,

This is a nice use case but I am sorry to say that your suggestion is not possible to do in Baker. Everything in Baker follows the Petrinet, so any functionality needs to be expressed when creating it. Now the "lock token" is a good idea except for the reprovisioning. Baker does not know the endstate of firing a sensory event from the Recipe. This is only known at runtime when executing a sensory event. At runtime we cannot change the Petrinet anymore so reproviding this token is impossible.

In recipe solution We already discussed how to do this in your recipe yourselves.

We can use an Event as token in this case. This can be provided once at the start of your Recipe. This Event should also be fire at the end of the non-concurrent path. Make this event be required at the start of your concurrent path. This should ensure it will only excute one at the same time.

There is one major problem with this solution. Ingredients used in this flow get overwritten by the other Sensory events. One idea we had was to reprovide the data at the start of the non-concurrent path. This would ensure other Sensory events would not overwrite the data when its in use. But this does not work if multiple Sensory events are in waiting. In this case when the next non-concurrent path can be started it will always use the latest data.

Dynamic recipe creation solution My solution is a little different. Instead of having a Recipe where we try do some tricks we create multiple paths in the Recipe. If I am not mistaked when baking the Recipe you know how many times the Sensory event needs to be fired. For each of theses you add the non-concurrent path seperatly to the recipe. Then once the Sensory-event if fired you have a sieve that triggers the correct path for that Sensory Event. How you do this I cannot say because that depends on your use case. Even better is if you create seperate Sensory events for these. But then you have to solve how to determine what Sensory event to fire.

I already know of a team that creates their Recipes dynamically. They have a base Recipe where they add Sensory Events and Interactions depending on the input for their process. I think this can also be done in this case.