kubesphere / notification-manager

K8s native notification management with multi-tenancy support
Apache License 2.0
226 stars 68 forks source link

notifycation-manager没有把告警消息转发到飞书群 #275

Closed ColeSu-n closed 2 days ago

ColeSu-n commented 2 days ago

Receiver

apiVersion: notification.kubesphere.io/v2beta2
kind: Receiver
metadata:
  name: global-receiver
  labels:
    type: global
    user: admin
  namespace: kubesphere-monitoring-federated
spec:
  feishu:
    chatbot:
      webhook:
        valueFrom:
          secretKeyRef:
            key: webhook
            name: global-receiver-secret
            namespace: kubesphere-monitoring-federated
    template: '{{ template "nm.feishu.text" . }}'
    tmplType: text
    tmplText:
      name: notification-manager-template
      namespace: kubesphere-monitoring-system

Secret

apiVersion: v1
kind: Secret
metadata:
  name: global-receiver-secret
  namespace: kubesphere-monitoring-federated
type: Opaque
data:
  webhook: "{{aaaaaaaaaaaaaaaaa}}" # 这里需替换为实际飞书Webhook地址的Base64编码值
  secret: "{{aaaaaaaaaaaaaaaaa}}" # 这里需替换为实际secret值的Base64编码值

nm

apiVersion: notification.kubesphere.io/v2beta1
kind: NotificationManager
metadata:
  annotations:
    meta.helm.sh/release-name: notification-manager
    meta.helm.sh/release-namespace: kubesphere-monitoring-system
  labels:
    app: notification-manager
    app.kubernetes.io/managed-by: Helm
  name: notification-manager
spec:
  affinity: {}
  defaultConfigSelector:
    matchLabels:
      type: default
  defaultSecretNamespace: kubesphere-monitoring-federated
  image: 'kubesphere/notification-manager:v2.3.0'
  imagePullPolicy: IfNotPresent
  nodeSelector: {}
  portName: webhook
  receivers:
    globalReceiverSelector:
      matchLabels:
        type: global
    options:
      email:
        deliveryType: bulk
        notificationTimeout: 5
      global:
        cluster: {{aaaaaaa}}
      slack:
        notificationTimeout: 5
      wechat:
        notificationTimeout: 5
    tenantKey: user
    tenantReceiverSelector:
      matchLabels:
        type: tenant
  replicas: 1
  resources:
    limits:
      cpu: 500m
      memory: 500Mi
    requests:
      cpu: 5m
      memory: 20Mi
  serviceAccountName: notification-manager-sa
  tolerations: []
  volumeMounts: []
  volumes: []

然后nm-deployment的日志是这样的

 level=error ts=2024-12-02T07:04:44.594071853Z caller=feishu.go:147 msg="FeishuNotifier: generate message error" error="template: :1:12: executing \"\" at <{{template \"nm.feishu.texts\" .}}>: template \"nm.feishu.texts\" not defined"

 level=error ts=2024-12-02T07:04:44.594091392Z caller=feishu.go:147 msg="FeishuNotifier: generate message error" error="template: :1:12: executing \"\" at <{{template \"nm.feishu.texts\" .}}>: template \"nm.feishu.texts\" not defined"

 level=error ts=2024-12-02T07:04:44.594189215Z caller=dispatcher.go:128 msg="Dispatcher: process alerts failed" seq=1557
 level=info ts=2024-12-02T07:04:53.172255424Z caller=controller.go:537 msg="Receiver changed" op=update name=global-receiver

alertmanager中告警是存在的,但是通知一直没有发布到飞书群的机器人里,需要从哪里排查问题

ColeSu-n commented 2 days ago

image image 另外想知道这个地方配置的通知渠道为什么也没有收到通知

ColeSu-n commented 2 days ago

已解决

benjaminhuo commented 2 days ago

已解决

什么问题

ColeSu-n commented 1 day ago

已解决

什么问题

能转发的,我吧所有Receiver和Config的cr删了重建,现在想想可能是配置url的base64有问题,测试下面几种方式都可以通知

apiVersion: notification.kubesphere.io/v2beta2
kind: Receiver
metadata:
  name: global-feishu-receiver
  labels:
    type: global
    user: admin
  namespace: kubesphere-monitoring-federated
spec:
  feishu:
    chatbot:
      webhook:
        valueFrom:
          secretKeyRef:
            key: webhook
            name: global-receiver-secret
            namespace: kubesphere-monitoring-federated
    template: '{{ template "nm.feishu.text" . }}'
    tmplType: text
    tmplText:
      name: notification-manager-template
      namespace: kubesphere-monitoring-system
apiVersion: v1
kind: Secret
metadata:
  name: global-receiver-secret
  namespace: kubesphere-monitoring-federated
type: Opaque
data:
  webhook: "" # 这里需替换为实际飞书Webhook地址的Base64编码值
  secret: "" # 这里需替换为实际secret值的Base64编码值
apiVersion: notification.kubesphere.io/v2beta2
kind: Receiver
metadata:
  name: global-webhook-receiver
  labels:
    type: global
    user: admin
spec:
  webhook:
    alertSelector:
      matchExpressions:
        - key: namespace
          operator: DoesNotExist
    enabled: true
    template: '{{ template "webhook.default.message" . }}'
    tmplText:
      name: notification-manager-template
      namespace: kubesphere-monitoring-system
    url: "http://${domain}/api/kwatch/alert"
    #service: []
    httpConfig: 
      tlsConfig: 
        insecureSkipVerify: true
apiVersion: notification.kubesphere.io/v2beta2
kind: Receiver
metadata:
  name: global-email-receiver
  labels:
    type: global
    user: admin
spec:
  email:
    enabled: true
    template: '{{ template "nm.default.html" . }}'
    subjectTemplate: '{{ template "nm.default.subject" .}}'
    tmplType: html
    tmplText:
      name: notification-manager-template
      namespace: kubesphere-monitoring-system
    to:
    - @gmail.com
    - @gmail.com
apiVersion: notification.kubesphere.io/v2beta2
kind: Config
metadata:
  name: default-email-config
  namespace: kubesphere-monitoring-federated
  labels:
    type: default
spec:
  email:
    hello: "hello"
    authIdentify: ""
    authPassword:
      value: "应用专用密码"
    authUsername: @gmail.com
    from: @gmail.com
    requireTLS: true
    smartHost:
      host: smtp.gmail.com
      port: 587
    tls: {}
ColeSu-n commented 1 day ago

已解决

什么问题

不过在页面上配置那些通知渠道还是没有生效,这个我确实还不知道原因 image image