druid-io / druid-operator

Druid Kubernetes Operator
Other
205 stars 93 forks source link

Why is there no MIDDLE_manager role ? #274

Open liubo-it opened 2 years ago

liubo-it commented 2 years ago

I use operator to deploy no middlerMangger or Indexer nodes on K8S

chrisguoado commented 2 years ago

I have the same doubt here. In the example tiny-cluster.yaml, there's no middlemanager nodes defined. But from the official documentation, I cannot find proof that middlemanager is an optional process. I then tested the example setup, ingesting from the sample http datasource and from a kafka cluster both worked well though.

AdheipSingh commented 2 years ago

the tiny-cluster is an example spec, to extend the spec you can add mm, indexer or whichever nodetype you prefer. a better example: https://gist.github.com/AdheipSingh/a87eb276badd2c614973d7aa56b07cbd

chrisguoado commented 2 years ago

Hi @AdheipSingh, thank you for the advice. Just trying to convince myself why I should configure a middlemanager in my cluster. Before seeing the tiny-cluster example, I always belived a mm is required. Also from the official doc:

The MiddleManager process is a worker process that executes submitted tasks. Middle Managers forward tasks to Peons that run in separate JVMs.

It didn't mention the mm process is optional, so looks like without it the tasks cannot be properly handled. But looking at the tiny-cluster example, I was wondering there could be a reason why mm was not included. And then from the actual testing result, looks like the mm is actually optional. Peons were created and the tasks were handled correctly.