Rework alertmanager to be installed the same way in SC and WC

[aarnq]

Is your feature request related to a problem? Please describe.

User alertmanager is not able to retain known alerts and silences across reboots since it runs without persistence, and it has no replicas that can keep things in memory while one replica reboots. This is because user alertmanager is completely separate compared to the ops alertmanager and never benefits from improvement made that the ops alertmanager gets.

Describe the solution you'd like

I want to see alertmanager reworked so that we have a common setup for both platform admins and application devs.

Both should be installed from the same chart with adaptions fit both admins and dev.

The admin alertmanager instance have had high availability and persistence for a long time, and other good configuration options and it is time we give the same for devs.

Also unifying the different instances will make it easier for us to configure it and ensure that the setup is good for both admin and devs.

Describe alternatives you've considered

Potentially also expose the option of running it HA.

Definition of done:

• Admin and dev alertmanager uses same setup (with some obvious differences to suite both)
• Admin and dev alertmanager has the standard options configurable, high availability by default and persistence by default.

[raviranjanelastisys]

Make it configurable but keep default at 2 replicas but don't hard code it.