jon-esperanza / silk-engine

Simple library for event processing with Kafka. Built on top of KafkaJS and inspired by Robinhood's Faust. Will be moved to Scala soon.
MIT License
0 stars 0 forks source link

Giving a Merchant multiple jobs to execute #13

Open jon-esperanza opened 2 years ago

jon-esperanza commented 2 years ago

:rocket: Feature Request

Is your feature request related to a problem? Please describe.

Can only assign a single job to a Merchant.

Describe the solution you'd like

Making Job property an array of Jobs. Make necessary changes to Merchant's execute function to iterate over multiple jobs and execute each one. This may involve figuring out how we want the jobs to be asynchronously called. (sequential or parallel)

Describe alternatives you've considered

Teachability, Documentation, Adoption, Migration Strategy

User has a SilkEngine subscribed to a topic that contains order messages. He has 2 jobs that he needs executed for the orderPlaced event messages. He can add both jobs to a single Merchant that will handle this for him.

jon-esperanza commented 2 years ago

Jobs cannot be fired in parallel due to our CentralizedSingleInstance that acts as a cache. The single data object would be in-use by another job therefore should be immutable. Jobs related to the same single instance must be fired in sequence to protect the integrity of each message consumed.

Although, if the same message object is used to fire multiple jobs --> if the user wants to, they can do parallel execution of jobs as long as they make sure not to mutate the data object to not affect other jobs. This could also be enforced by the Silk Engine by passing a copy of the artificially immutable object to each job (which might affect performance overhead).

Typescript allows objects to be casted as immutable by using Readonly<T>