search

seanttaylor / hypertoast

A Node project implementing HATEOS with an API for a smart toaster
0 stars 0 forks source link

Infinite request loop to HyperToast application instances when Multigrain receives more requests than available instances #2

Open seanttaylor opened 1 year ago

seanttaylor commented 1 year ago

When Multigrain receives more requests than there are application instances to serve the requests, the desired behavior is for Multigrain to continue polling the registered application instances until an instance becomes available to route a pending request to.

The observed behavior is that the Multigrain broker enters an infinite loop at some point and continues to make requests over and over again to application instances (i.e. the broker never finishes its workload). Additionally the requests being made are old requests, (i.e. requests that have already been processed)

Possible solution: is to pull the incoming messages from the Kafka stream and place them into an in memory object (e.g. pendingRequests). When a HyperToast application instance publishes the toaster-off event use the SmartRouter to locate the specific instance that published the toaster-off event (i.e. the device that just finished processing) and route the request to that instance.

Refactor HTSmartRouterPlugin.findAvailableToaster. When called with a URN method should return a specified application instance. When called without a URN, findAvailableToaster iterates over all registered toasters and returns the first available application instance.

seanttaylor commented 1 year ago

I'm convinced this has to do with the ability of Kafka to reprocess old messages from the beginning when the Kafka broker goes down for whatever reason.