search

cpnr / computing

0 stars 0 forks source link

grafana 패널에서 CPU사용량 등 세부 정보 표시 문제 #45

Closed jhgoh closed 6 months ago

jhgoh commented 6 months ago

Telegraf에서 보내는 정보들이 표시되지 않는 문제가 있음.

image

influxdb에 문제가 있는 것으로 보임

● telegraf.service - Telegraf
     Loaded: loaded (/lib/systemd/system/telegraf.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2024-04-08 16:27:30 KST; 1 month 13 days ago
       Docs: https://github.com/influxdata/telegraf
   Main PID: 1485096 (telegraf)
      Tasks: 35 (limit: 154082)
     Memory: 69.5M
     CGroup: /system.slice/telegraf.service
             ├─1485096 /usr/bin/telegraf -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d
             └─1485118 /usr/bin/dbus-daemon --syslog --fork --print-pid 4 --print-address 6 --session

May 22 15:50:09 hep.khu.ac.kr telegraf[1485096]: 2024-05-22T06:50:09Z E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
May 22 15:51:09 hep.khu.ac.kr telegraf[1485096]: 2024-05-22T06:51:09Z W! [outputs.influxdb_v2] Metric buffer overflow; 142 metrics have been dropped
May 22 15:51:09 hep.khu.ac.kr telegraf[1485096]: 2024-05-22T06:51:09Z E! [outputs.influxdb_v2] When writing to [http://hep.khu.ac.kr:30086]: 500 Internal Server Error: internal error: unexpected error w>
May 22 15:51:09 hep.khu.ac.kr telegraf[1485096]: 2024-05-22T06:51:09Z E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
May 22 15:52:09 hep.khu.ac.kr telegraf[1485096]: 2024-05-22T06:52:09Z W! [outputs.influxdb_v2] Metric buffer overflow; 142 metrics have been dropped
May 22 15:52:09 hep.khu.ac.kr telegraf[1485096]: 2024-05-22T06:52:09Z E! [outputs.influxdb_v2] When writing to [http://hep.khu.ac.kr:30086]: 500 Internal Server Error: internal error: unexpected error w>
May 22 15:52:09 hep.khu.ac.kr telegraf[1485096]: 2024-05-22T06:52:09Z E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
May 22 15:53:09 hep.khu.ac.kr telegraf[1485096]: 2024-05-22T06:53:09Z W! [outputs.influxdb_v2] Metric buffer overflow; 142 metrics have been dropped
May 22 15:53:09 hep.khu.ac.kr telegraf[1485096]: 2024-05-22T06:53:09Z E! [outputs.influxdb_v2] When writing to [http://hep.khu.ac.kr:30086]: 500 Internal Server Error: internal error: unexpected error w>
May 22 15:53:09 hep.khu.ac.kr telegraf[1485096]: 2024-05-22T06:53:09Z E! [agent] Error writing to outputs.influxdb_v2: failed to send metrics to any configured server(s)
jhgoh commented 6 months ago

일단 kubernetes에서 influxdb manifest 삭제 후 재적용.

kubectl delete -f influxdb.yaml
kubectl apply -f influxdb.yaml
jhgoh commented 6 months ago

다시 정보가 나타나는 것으로 보아 정상 동작함. Disk full문제 #44 와 관련된 것으로 보임.

image