Open liubo-it opened 2 years ago
I have the same doubt here. In the example tiny-cluster.yaml
, there's no middlemanager nodes defined. But from the official documentation, I cannot find proof that middlemanager is an optional process. I then tested the example setup, ingesting from the sample http datasource and from a kafka cluster both worked well though.
the tiny-cluster is an example spec, to extend the spec you can add mm, indexer or whichever nodetype you prefer. a better example: https://gist.github.com/AdheipSingh/a87eb276badd2c614973d7aa56b07cbd
Hi @AdheipSingh, thank you for the advice. Just trying to convince myself why I should configure a middlemanager in my cluster. Before seeing the tiny-cluster example, I always belived a mm is required. Also from the official doc:
The MiddleManager process is a worker process that executes submitted tasks. Middle Managers forward tasks to Peons that run in separate JVMs.
It didn't mention the mm process is optional, so looks like without it the tasks cannot be properly handled. But looking at the tiny-cluster example, I was wondering there could be a reason why mm was not included. And then from the actual testing result, looks like the mm is actually optional. Peons were created and the tasks were handled correctly.
I use operator to deploy no middlerMangger or Indexer nodes on K8S