ibmresilient / resilient-python-api

Python Library for the IBM SOAR REST API, a Python SDK for developing Apps for IBM SOAR and more...
https://ibm.biz/soar-python-docs
MIT License
39 stars 28 forks source link

Usage of num_workers in separate resilient-circuits instances #30

Closed poleshe closed 1 year ago

poleshe commented 1 year ago

Greetings, team!

Not an issue, apologies, but a question.

How does num_workers work if we do not have a central resilient-circuits daemon, but instead an instance for each integration?

Similar to the AppHost containing each integration in it's own container. Does it mean that if we assign num_workers to 50, we can consume 50 events from the queue and handle them in paral·lel? Or it does not apply in singular instaces?

We tried looking for documentation but couldn't find any

Thanks for your help! Have a nice day!

bobleckelibm commented 1 year ago

Hi @poleshe -- your idea of consuming 50 events in parallel is about correct. Let's say, for example, that you have one app with one function with num_workers=50 running in an isolated app host environment. If you somehow trigger that one function 50 times exactly at the same time, they should all process in relatively close to parallel time and return their results at or around the same time (assuming same run time for each instance of the function). Now this is all theoretical... There may be hardware limits that might impede the processing happening all in the same time, however, the idea still holds. In the case that you have an app running in an isolated (read: AppHost) environment, you should be able to up num_workers and see an increase in parallel processing.

Marking this as closed for now, but let me know if you have further questions. Thanks!