Manage Multiple environments with the Release Manager

[SHoen]

Is your feature request related to a problem? Please describe. Assume you have multiple productive deployments in different regions. You want to release a version independent to the QA and to the productive stage on another cluster. You need different configuration in the regions.

Describe the solution you'd like On Dev env, you could create multiple deployments with different config. You can "move" each of them to a QA Environment. After approval from the QA and relevant stakeholders (also regional ones) you can "move" the version to the productive one.

Describe alternatives you've considered Alternative 1: One "global QA env"

• Constraint: Allows only sequential releases in regions. First release could block release of other region.

Alternative 2: Multiple configurations for the regions and one Release Manager.

• Constraint: Rollout simultaneously. Regional approvals could cause delays

@michaelsauter @clemensutschig

[michaelsauter]

@SHoen Thanks for the detailed write-up! As we discussed, we need to collect a few more use-cases I think to find a solution that works well enough for everyone!

FYI @metmajer

[SHoen]

@michaelsauter @metmajer @clemensutschig Is a feature like this in consideration with more then 3 environments for 4.x?

[michaelsauter]

I am exploring this at the moment, and the roadmap lists this for 4.x . What do you mean with more then 3 environments? Are you referring to more "stages" like DEV, QA, PROD - or are you referring to have multiple PROD deployments?

[SHoen]

I meant stages like DEV -> TEST -> QA -> PROD. Three PROD deployments in different regions should be sufficient.

[michaelsauter]

@SHoen Ok, interesting! I would like to hear the rationale behind that, and also how you would treat these additional environments from a validation perspective? Would TEST be basically another DEV in that regard?

We won't limit the number of PROD deployments, neither in terms of clusters, nor namespaces (e.g. one could deploy to multiple namespaces in the same cluster).

[SHoen]

Yes of course. I should explain the purpose of the "stages".

DEV: Is used for Development. The developers can have a look to the outcome of their work and do testing.

TEST would be another DEV environment. It's used for Testautomation. The automated tests can inject and delete all data without blocking any Development Team or other Stakeholder.

QA would be the QA environment in validation terms. You can migrate the data from the PROD instance. If you have the same number like PROD deployments it could be used for User Acceptance Tests on a regional level and get approval from local stakeholders.

PROD: Regional deployments for productive use.

[michaelsauter]

@SHoen Cool, got it.

One more question: it sounds like you'd also want to separate the PROD deployments. Meaning - whenever regional QA gives green light, you want to move to the corresponding regional PROD. Is that correct?

[michaelsauter]

@SHoen Actually - would a separate DEV namespace be enough? Do you really need a separate "stage"?

[SHoen]

Regarding separate PROD deployments: Yes exactly, that is correct!

Regarding separate DEV namespace. How would a setup with separate DEV namespace look like? The idea was to create the (immutable) artifacts and test them before releasing to QA/PROD.

If this "additional stage" causes a lot of effort for you we can go with an alternative setup like seperate DEV namespace.

[michaelsauter]

Ok. Right now I'm not thinking about effort yet - I just want to figure out what the need is, and for each need, to figure out if we need a separate concept for it or can achieve that (or 90% of it or so) with functionality we have elsewhere. I think I need to think about multiple stages a bit more.

[michaelsauter]

@SHoen Please see my proposal for the component pipeline at https://github.com/opendevstack/ods-jenkins-shared-library/pull/535. It would be interesting to know if this would work for you technically - even if it is not the release manager (= without documents). One way to have separate QA/PROD deployments by region would be to point to different ConfigMap resources based on the branch ...

[SHoen]

@michaelsauter I had a look at #535. For me the main advantages of the release manager are:

1. Create a single version for the whole bundle for the systems and tag the state in all repos.
2. Build and Deployment are separated. Devs don't need to focus on deployment once it built and tested.
3. Deployments are specified in the "Deployment Tool" and not in all components which could lead to inconsistencies.
4. Auto-generated documents are a plus. (For me not really necessary at the moment)

Technically spoken we can move away from the Release-Manager. I would prefer a central config map to maintain the deployments but I would miss the "bundle-version" and independent deployment without merging branches of all related components in the right order.

[michaelsauter]

@SHoen thanks for the input! Makes sense for me. I have doubts though if (3) is really good - it's quite limiting that deployment cannot be customised by each component.

In any case, I don't want to suggest to avoid the release manager, I just wanted to see whether your concrete use case aligns with #535. That said, right now I'm not sure if #535 will move forward as is.

[SHoen]

@michaelsauter Thanks for your fast response! May I ask that you share your doubts? Probably I am not aware of a pitfall.

[michaelsauter]

It might work well for you. But it's limited in that people have very little control over the deployment. On the one hand this is great, because if you use the deployment as defined in the pipeline, you do not need to verify this in regulated environments. On the other hand, it's bad if you cannot get your use case to work with the pre-defined deployment step. Then you have no options around it. Further, something that might work in the component pipeline (because there you can control everything in the end) might not work in the orchestration pipeline.

[serverhorror]

Note to self:

• serverhorror/ods-jenkins-shared-library:configReleaseNamespaces