unaccessible prometheus ui after reboot cluster

[aki0000]

It's unaccessible prometheus UI (http://192.168.100.111:9090/) after reboot cluster.

Services

 k get svc -A
NAMESPACE     NAME             TYPE           CLUSTER-IP       EXTERNAL-IP       PORT(S)                  AGE
default       kubernetes       ClusterIP      10.96.0.1        <none>            443/TCP                  21d
kube-system   kube-dns         ClusterIP      10.96.0.10       <none>            53/UDP,53/TCP,9153/TCP   21d
kube-system   metrics-server   ClusterIP      10.99.20.179     <none>            443/TCP                  5d13h
monitoring    grafana-lb       LoadBalancer   10.98.40.167     192.168.100.112   3000:31187/TCP           5d14h
monitoring    prometheus       ClusterIP      10.110.248.149   <none>            9090/TCP                 15s
monitoring    prometheus-lb    LoadBalancer   10.96.153.75     192.168.100.111   9090:32222/TCP           15s

Pods

 k get pods -A
NAMESPACE        NAME                             READY   STATUS         RESTARTS   AGE
kube-system      coredns-74ff55c5b-gqsnl          1/1     Running        14         21d
kube-system      coredns-74ff55c5b-kg56v          1/1     Running        14         21d
kube-system      etcd-ras01                       1/1     Running        15         21d
kube-system      kube-apiserver-ras01             1/1     Running        16         21d
kube-system      kube-controller-manager-ras01    1/1     Running        16         21d
kube-system      kube-flannel-ds-arm-96ssz        1/1     Running        16         21d
kube-system      kube-flannel-ds-arm-xdjzf        1/1     Running        11         21d
kube-system      kube-flannel-ds-arm-zr6jg        1/1     Running        16         21d
kube-system      kube-proxy-6j5vk                 1/1     Running        11         21d
kube-system      kube-proxy-fggtl                 1/1     Running        11         21d
kube-system      kube-proxy-mx6rz                 1/1     Running        14         21d
kube-system      kube-scheduler-ras01             1/1     Running        16         21d
kube-system      metrics-server-c54cc79c6-fp7cl   1/1     Running        1          7h16m
kube-system      metrics-server-c54cc79c6-nf5v5   0/1     NodeAffinity   0          5d13h
metallb-system   controller-65db86ddc6-qhlr2      1/1     Running        0          7h34m
metallb-system   speaker-ks4g2                    1/1     Running        1          5d14h
metallb-system   speaker-n76g4                    1/1     Running        1          5d14h
metallb-system   speaker-rjwhs                    1/1     Running        3          5d14h
monitoring       grafana-5b46cb5f6-bbm97          1/1     Running        0          7h34m

Deploy

# k get deploy -A
NAMESPACE        NAME             READY   UP-TO-DATE   AVAILABLE   AGE
kube-system      coredns          2/2     2            2           21d
kube-system      metrics-server   0/1     1            0           5d13h
metallb-system   controller       0/1     1            0           5d14h
monitoring       grafana          0/1     1            0           5d14h
monitoring       prometheus       0/1     0            0           2m55s

[aki0000]

Need to configure RBCA for deployment by helm? https://christina04.hatenablog.com/entry/helm-basic-usage

[aki0000]

Created ClusterRoleBinding and ServiceAccount for Promehetus, and deployed it in the cluster. Please check below.

Also, I need to consider RBCA as per namespace in the cluster like below.... I think. Monitoring -> SA1 Kube-System -> SA2 another -> SA2

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: {{ .Release.Name }}
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: {{ .Release.Name }}-admin
subjects:
- kind: ServiceAccount
  name: {{ .Release.Name }}
  namespace: {{ .Release.Namespace }}
----
apiVersion: v1
kind: ServiceAccount
metadata:
  name: {{ .Release.Name }}
  namespace: {{ .Release.Namespace }}