How to configure alertmanager service from victoria-metrics-k8s-stack as NodePort

[enp]

Can't find any way to configure alertmanager service from victoria-metrics-k8s-stack as NodePort

Found serviceSpec in https://docs.victoriametrics.com/operator/api.html#vmalertmanagerspec and https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-k8s-stack/templates/victoria-metrics-operator/alertmanager/vmalertmanager.yaml#L29 but can't understand what include "victoria-metrics-k8s-stack. means and how to set spec from values.yaml

Or maybe more simple solution exists?

[enp]

Temporary fixed local file templates/victoria-metrics-operator/alertmanager/vmalertmanager.yaml in chart like so:

apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAlertmanager
metadata:
...
spec:
{{ include "victoria-metrics-k8s-stack.alertmanagerSpec" . | indent 2}}
  serviceSpec:
    spec:
      type: NodePort
{{- end }}

So I see updated resource in helm upgrade ... --dry-run --debug, but no effect in kubectl describe svc vmalertmanager-vm-stack-victoria-metrics-k8s-stack -n monitoring - type is still ClusterIP. Bug in operator or I'm using it in wrong way?

[enp]

Looks like it works different in different cases

I can create alertmanager with service type=NodePort directly with VMAlertmanager resource:

$ helm install --create-namespace --namespace monitoring vm-operator vm/victoria-metrics-operator
$ cat << EOF | kubectl apply -n monitoring -f -
apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAlertmanager
metadata:
  name: vmalertmanager
spec:
  replicaCount: 1
  selectAllByDefault: true
  serviceSpec:
    spec:
      type: NodePort
EOF
$ kubectl get svc -n monitoring
NAME                                               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
vm-operator-victoria-metrics-operator              ClusterIP   10.96.210.137   <none>        8080/TCP,443/TCP             3m41s
vmalertmanager-vmalertmanager                      ClusterIP   None            <none>        9093/TCP,9094/TCP,9094/UDP   5s
vmalertmanager-vmalertmanager-additional-service   NodePort    10.96.175.41    <none>        9093:31451/TCP               5s

But I can't do the same with helm:

$ cat << EOF | helm install --create-namespace --namespace monitoring vm-stack vm/victoria-metrics-k8s-stack -f -
alertmanager:
  spec:
    serviceSpec:
      spec:
        type: NodePort
  ingress:
    enabled: true
    annotations:
      ingress.alb.yc.io/subnets: b0c4bu9bcpepnsvcuvu8
      ingress.alb.yc.io/internal-alb-subnet: b0c4bu9bcpepnsvcuvu8
      ingress.alb.yc.io/internal-ipv4-address: 10.92.2.100
      ingress.alb.yc.io/group-name: alb-internal
    hosts:
      - alerts.kube.local
    tls:
      - secretName: yc-certmgr-cert-id-fpqrtitjghgn1m2c6rco
        hosts:
          - alerts.kube.local
EOF
$ kubectl get svc -n monitoring
NAME                                           TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)            AGE
vm-stack-grafana                               ClusterIP   10.96.149.213   <none>        80/TCP             97s
vm-stack-kube-state-metrics                    ClusterIP   10.96.231.170   <none>        8080/TCP           97s
vm-stack-prometheus-node-exporter              ClusterIP   10.96.177.186   <none>        9100/TCP           97s
vm-stack-victoria-metrics-operator             ClusterIP   10.96.203.245   <none>        8080/TCP,443/TCP   97s
vmagent-vm-stack-victoria-metrics-k8s-stack    ClusterIP   10.96.170.37    <none>        8429/TCP           74s
vmalert-vm-stack-victoria-metrics-k8s-stack    ClusterIP   10.96.234.58    <none>        8080/TCP           75s
vmsingle-vm-stack-victoria-metrics-k8s-stack   ClusterIP   10.96.218.214   <none>        8429/TCP           75s

But even in this case VMAlertmanager resource looks similar to directly created:

$ kubectl get vmalertmanager -n monitoring -o yaml
apiVersion: v1
items:
- apiVersion: operator.victoriametrics.com/v1beta1
  kind: VMAlertmanager
  metadata:
    annotations:
      meta.helm.sh/release-name: vm-stack
      meta.helm.sh/release-namespace: monitoring
    creationTimestamp: "2022-04-28T17:53:19Z"
    finalizers:
    - apps.victoriametrics.com/finalizer
    generation: 2
    labels:
      app.kubernetes.io/component: victoria-metrics-k8s-stack-alertmanager
      app.kubernetes.io/instance: vm-stack
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: victoria-metrics-k8s-stack
      app.kubernetes.io/version: 1.76.1
      helm.sh/chart: victoria-metrics-k8s-stack-0.8.1
    name: vm-stack-victoria-metrics-k8s-stack
    namespace: monitoring
    resourceVersion: "958076"
    selfLink: /apis/operator.victoriametrics.com/v1beta1/namespaces/monitoring/vmalertmanagers/vm-stack-victoria-metrics-k8s-stack
    uid: 0bf06fed-7b70-46fd-9f8d-1dcc4d89fe38
  spec:
    configMaps:
    - vm-stack-victoria-metrics-k8s-stack-alertmanager-monzo-tpl
    configSecret: vm-stack-victoria-metrics-k8s-stack-alertmanager
    image:
      tag: v0.23.0
    resources: {}
    routePrefix: /
    selectAllByDefault: true
    serviceSpec:
      metadata: {}
      spec:
        type: NodePort
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

So why alertmanager with service type=NodePort can't be created from VMAlertmanager via helm but can be created by VMAlertmanager directly?

[f41gh7]

It seems, that spec for alertmanager is not rendered correctly at _helpers https://github.com/VictoriaMetrics/helm-charts/blob/master/charts/victoria-metrics-k8s-stack/templates/_helpers.tpl#L160

@k1rk PTAL

[Haleygo]

@enp Hello, is this still a problem to you? I tested two ways in my env, the service type looks ok.

1. add service spec under .alertmanger.spec in values.yaml

2. add service spec directly in file templates/victoria-metrics-operator/alertmanager/vmalertmanager.yaml

[hagen1778]

Closing this issue as inactive. @enp feel free to reopen if still is a problem for you