@romain-fontugne I was planning to create an entire docker environment for our kafka server. But what I have observed is we have around 6-7 producers and a bunch of consumers in psql/consumers. Do you have any existing approach as to how to club these producers and consumers? I have 2 options:
Creating separate containers for each producer and consumer which will allow us to scale specific components if they generate data with high velocity and have fine-grained control over the components. But this can create a little bit of chaos as well since we'll have to deploy 20+ containers.
If we want a simpler deployment and management experience, we could combine all of the producers into one container and all of the consumer into another container. Just the fact I feel is it would limit our ability to scale individual components independently.
But entirely your call. We can move forward accordingly.
@romain-fontugne I was planning to create an entire docker environment for our kafka server. But what I have observed is we have around 6-7 producers and a bunch of consumers in psql/consumers. Do you have any existing approach as to how to club these producers and consumers? I have 2 options:
Creating separate containers for each producer and consumer which will allow us to scale specific components if they generate data with high velocity and have fine-grained control over the components. But this can create a little bit of chaos as well since we'll have to deploy 20+ containers.
If we want a simpler deployment and management experience, we could combine all of the producers into one container and all of the consumer into another container. Just the fact I feel is it would limit our ability to scale individual components independently.
But entirely your call. We can move forward accordingly.