Open Zelldon opened 2 years ago
@chDame FYI
Instead of splitting the importers into a separate deployment, we could configure each replica's importer to work on a different chunk of data. However, that would likely require a stateful set to control pod identity, i.e. the ordinal of the set member. The downside of it is that this configuration would be pretty static, i.e. reconfiguration requires pod restarts, and therefore not work with the HPA.
Or only importing on pod 0 would even be simpler.
Currently we deploy for the operate service one deployment, which contains the importer, archiver and webapp. Ideally we split this up, to scale it individually and be more fault tolerant.
See the docs for more details https://docs.camunda.io/docs/self-managed/operate-deployment/importer-and-archiver/