Historically, we've evolved listen_to to use configurable pre-defined ThreadPools, which contributed to stability, also allowing to segregate pools for different use cases (db/sb/app) as opposed to unbound threads that used to be in place.
Motivation / Use cases:
Currently, in the application, listen_to events since they're running in a ThreadPool, they'll get fetched/started in order but since the concurrency is greater than 1, eventually threads get suspended and one of the executions might end up being late finishing later than another recent one resulting in out of order execution of events, which have contributed to these bugs: https://github.com/kytos-ng/of_core/issues/118, https://github.com/kytos-ng/pathfinder/issues/34. In this case, it's clear that certain handlers where the order of the events matter, that don't do much IO could benefit from a single threadpool worker ensuring the order and only fetching a new task when it's done. So, it's worth exploring the option of parametrizing pool="fifo_dynamic" or something similar where an exclusive ThreadPool with a single worker would be used for that handler.
Main benefit:
Minimize room for race conditions when certain handlers are susceptible to and can benefit from a using just a single exclusive thread processing, consequently minimizing bugs and leading to more stability.
Another altenative that we've been liveraging is the alisten_toasync handler, which can also help since it has cooperative preemption and it's easier to control if the task will yield or not and deal with or protect any concurrent rw, but in we still have a synchronous db driver pymongo and we can't always reach out to alisten_to yet, so having this option to use a dynamic single worker fifo handler would be great, which also wouldn't break any compatibility and then application handlers could start to use it when appropriate.
Historic context:
listen_toto use configurable pre-defined ThreadPools, which contributed to stability, also allowing to segregate pools for different use cases (db/sb/app) as opposed to unbound threads that used to be in place.Motivation / Use cases:
listen_toevents since they're running in a ThreadPool, they'll get fetched/started in order but since the concurrency is greater than 1, eventually threads get suspended and one of the executions might end up being late finishing later than another recent one resulting in out of order execution of events, which have contributed to these bugs: https://github.com/kytos-ng/of_core/issues/118, https://github.com/kytos-ng/pathfinder/issues/34. In this case, it's clear that certain handlers where the order of the events matter, that don't do much IO could benefit from a single threadpool worker ensuring the order and only fetching a new task when it's done. So, it's worth exploring the option of parametrizingpool="fifo_dynamic"or something similar where an exclusive ThreadPool with a single worker would be used for that handler.Main benefit:
Another altenative that we've been liveraging is the
alisten_toasynchandler, which can also help since it has cooperative preemption and it's easier to control if the task will yield or not and deal with or protect any concurrent rw, but in we still have a synchronous db driverpymongoand we can't always reach out toalisten_toyet, so having this option to use a dynamic single worker fifo handler would be great, which also wouldn't break any compatibility and then application handlers could start to use it when appropriate.