search

honzaap / Systemizer

A system design tool that allows you to simulate data flow of distributed systems.
https://honzaap.github.io/Systemizer/
GNU General Public License v3.0
1.39k stars 89 forks source link

Consumer/processor component #42

Open Shmuma opened 3 years ago

Shmuma commented 3 years ago

Current set of components are good to design simple web-based apps, but in practice there are more cases. For example, almost immediately I missed “consumer” or “processor” - worker which consumes MQ messages and transforms them. In general they can do multiple requests, not just transformations, for example it can consume “crawling requests” which contains domain names, perform the crawling of the website (HTTP requests) and push results into another MQ queue.

honzaap commented 3 years ago

Sounds good, I'll look into it.

honzaap commented 3 years ago

Hi, I'm reading more into this now and can't kinda understand what would make this "processor" component so unique. Right now, you can connect API component as a consumer, which can then send requests to another MQ, I don't see the differece.

Lockhead commented 3 years ago

For example how could I build a 3 way connection as processor that consumes a queue as input, requests to other services to enrich and then pushes the result into another queue