search

netbox-community / netbox-chart

A Helm chart for NetBox
https://netbox.readthedocs.io/
Apache License 2.0
258 stars 153 forks source link

Netbox Pod crashing #316

Closed tbartschat closed 2 months ago

tbartschat commented 2 months ago

General settings

Kubernetes Version: v1.30.0 OS Version: Ubuntu 22.04.4 LTS Netbox Helm Chart Version: 5.0.0-beta.78 Netbox appVersion: v4.0.9

In some cases the netbox pod is crashing. Let me describe two cases.

1st case

While running python import scripts in background (one by one, no bulk create) and parallel editing device status or anything else via Webgui (on different device), the pod chrashed and restarts.

Python VENV: 3.10.12 pynetbox: 7.3.4

2nd case

When I want to delete many devices (approx 400) via WebGui and performing a bulk delete request the netbox pod is also crashing. Service is not given and after the crash the devices are not deleted.

I already tried with to deploy 2 netbox pods running but the result is the same.

Any ideas or suggestions here what to do?

tbartschat commented 2 months ago

Please see the latest log:

Defaulted container "netbox" out of: netbox, init-dirs (init)
🧬 loaded config '/etc/netbox/config/configuration.py'
🧬 loaded config '/etc/netbox/config/extra.py'
🧬 loaded config '/etc/netbox/config/logging.py'
🧬 loaded config '/etc/netbox/config/plugins.py'
💡 Superuser Username: root, E-Mail: matthias.wonneberger1@vodafone.com
🧬 loaded config '/etc/netbox/config/configuration.py'
🧬 loaded config '/etc/netbox/config/extra.py'
🧬 loaded config '/etc/netbox/config/logging.py'
🧬 loaded config '/etc/netbox/config/plugins.py'
✅ Initialisation is done.
2024/08/22 09:44:45 [warn] 7#7 Unit is running unprivileged, then it cannot use arbitrary user and group.
2024/08/22 09:44:45 [info] 7#7 unit 1.32.0 started
2024/08/22 09:44:45 [info] 102#102 discovery started
2024/08/22 09:44:45 [notice] 102#102 module: python 3.11.6 "/usr/lib/unit/modules/python3.11.unit.so"
2024/08/22 09:44:45 [info] 7#7 controller started
2024/08/22 09:44:45 [notice] 7#7 process 102 exited with code 0
2024/08/22 09:44:45 [info] 104#104 router started
2024/08/22 09:44:45 [info] 104#104 OpenSSL 3.0.10 1 Aug 2023, 300000a0
2024/08/22 09:44:45 [info] 105#105 "netbox" prototype started
2024/08/22 09:44:45 [info] 106#106 "netbox" application started
2024/08/22 09:44:53 [info] 177#177 "netbox" application started
⚙️ Applying configuration from /etc/unit/nginx-unit.json
✅ Unit configuration loaded successfully
2024/08/22 09:44:53 [notice] 7#7 process 100 exited with code 0
10.244.2.85 - - [22/Aug/2024:09:44:59 +0000] "GET /metrics HTTP/1.1" 200 18415 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:44:59 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
2024/08/22 09:45:09 [info] 201#201 "netbox" application started
10.244.4.245 - - [22/Aug/2024:09:45:09 +0000] "GET /metrics HTTP/1.1" 200 24493 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:45:10 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:45:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:45:20 +0000] "GET /metrics HTTP/1.1" 200 24492 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:45:30 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:45:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:45:39 +0000] "GET /metrics HTTP/1.1" 200 21829 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:45:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:45:50 +0000] "GET /metrics HTTP/1.1" 200 24492 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:45:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:46:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:46:09 +0000] "GET /metrics HTTP/1.1" 200 24490 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:46:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:46:20 +0000] "GET /metrics HTTP/1.1" 200 24523 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:46:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:46:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:46:39 +0000] "GET /metrics HTTP/1.1" 200 21832 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:46:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:46:50 +0000] "GET /metrics HTTP/1.1" 200 24493 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:46:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:47:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:47:09 +0000] "GET /metrics HTTP/1.1" 200 24559 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:47:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:47:20 +0000] "GET /metrics HTTP/1.1" 200 24598 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:47:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:47:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:47:39 +0000] "GET /metrics HTTP/1.1" 200 24597 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:47:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:47:50 +0000] "GET /metrics HTTP/1.1" 200 24606 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:47:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:48:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:48:09 +0000] "GET /metrics HTTP/1.1" 200 24493 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:48:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:48:20 +0000] "GET /metrics HTTP/1.1" 200 24584 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:48:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:48:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:48:39 +0000] "GET /metrics HTTP/1.1" 200 24607 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:48:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:48:50 +0000] "GET /metrics HTTP/1.1" 200 24492 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:48:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:49:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:49:09 +0000] "GET /metrics HTTP/1.1" 200 24608 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:49:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:49:20 +0000] "GET /metrics HTTP/1.1" 200 24489 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:49:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:49:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:49:39 +0000] "GET /metrics HTTP/1.1" 200 24618 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:49:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:49:50 +0000] "GET /metrics HTTP/1.1" 200 24608 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:49:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:50:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:50:09 +0000] "GET /metrics HTTP/1.1" 200 24620 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:50:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:50:20 +0000] "GET /metrics HTTP/1.1" 200 24511 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:50:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:50:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:50:39 +0000] "GET /metrics HTTP/1.1" 200 24620 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:50:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:50:50 +0000] "GET /metrics HTTP/1.1" 200 24608 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:50:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:51:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:51:09 +0000] "GET /metrics HTTP/1.1" 200 24628 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:51:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:51:20 +0000] "GET /metrics HTTP/1.1" 200 24575 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:51:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:51:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:51:39 +0000] "GET /metrics HTTP/1.1" 200 24620 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:51:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:51:50 +0000] "GET /metrics HTTP/1.1" 200 24597 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:51:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:52:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:52:09 +0000] "GET /metrics HTTP/1.1" 200 24585 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:52:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:52:20 +0000] "GET /metrics HTTP/1.1" 200 24626 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:52:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:52:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:52:39 +0000] "GET /metrics HTTP/1.1" 200 24600 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:52:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:52:50 +0000] "GET /metrics HTTP/1.1" 200 24626 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:52:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:53:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:53:09 +0000] "GET /metrics HTTP/1.1" 200 24641 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:53:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:53:20 +0000] "GET /metrics HTTP/1.1" 200 24606 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:53:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:53:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:53:39 +0000] "GET /metrics HTTP/1.1" 200 24628 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:53:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:53:50 +0000] "GET /metrics HTTP/1.1" 200 24629 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:53:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:54:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:54:09 +0000] "GET /metrics HTTP/1.1" 200 24642 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:54:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:54:20 +0000] "GET /metrics HTTP/1.1" 200 24629 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:54:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:54:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:54:39 +0000] "GET /metrics HTTP/1.1" 200 24628 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:54:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:54:50 +0000] "GET /metrics HTTP/1.1" 200 24627 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:54:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:55:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:55:09 +0000] "GET /metrics HTTP/1.1" 200 24628 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:55:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:55:20 +0000] "GET /metrics HTTP/1.1" 200 24629 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:55:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:55:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:55:39 +0000] "GET /metrics HTTP/1.1" 200 24627 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:55:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:55:50 +0000] "GET /metrics HTTP/1.1" 200 24629 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:55:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:56:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:56:09 +0000] "GET /metrics HTTP/1.1" 200 24629 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:56:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:56:20 +0000] "GET /metrics HTTP/1.1" 200 24626 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:56:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:56:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:56:39 +0000] "GET /metrics HTTP/1.1" 200 24628 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:56:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:56:50 +0000] "GET /metrics HTTP/1.1" 200 24629 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:56:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:57:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:57:09 +0000] "GET /metrics HTTP/1.1" 200 24627 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:57:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:57:20 +0000] "GET /metrics HTTP/1.1" 200 24629 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:57:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:57:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:57:39 +0000] "GET /metrics HTTP/1.1" 200 24642 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:57:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:57:50 +0000] "GET /metrics HTTP/1.1" 200 24626 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:57:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:58:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:58:09 +0000] "GET /metrics HTTP/1.1" 200 24627 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:58:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:58:20 +0000] "GET /metrics HTTP/1.1" 200 24626 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:58:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:58:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:58:39 +0000] "GET /metrics HTTP/1.1" 200 24629 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:58:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:58:50 +0000] "GET /metrics HTTP/1.1" 200 24627 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:58:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:59:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:59:09 +0000] "GET /metrics HTTP/1.1" 200 24626 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:59:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:59:20 +0000] "GET /metrics HTTP/1.1" 200 24640 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:59:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:09:59:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:09:59:39 +0000] "GET /metrics HTTP/1.1" 200 24628 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:59:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:09:59:50 +0000] "GET /metrics HTTP/1.1" 200 24626 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:09:59:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:00:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:00:09 +0000] "GET /metrics HTTP/1.1" 200 24637 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:00:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:00:20 +0000] "GET /metrics HTTP/1.1" 200 24639 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:00:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:00:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:00:39 +0000] "GET /metrics HTTP/1.1" 200 24626 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:00:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:00:50 +0000] "GET /metrics HTTP/1.1" 200 24639 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:00:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
🧬 loaded config '/etc/netbox/config/configuration.py'
🧬 loaded config '/etc/netbox/config/extra.py'
🧬 loaded config '/etc/netbox/config/logging.py'
🧬 loaded config '/etc/netbox/config/plugins.py'
10.244.1.1 - - [22/Aug/2024:10:01:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
2024/08/22 10:01:06 [notice] 105#105 app process 201 exited with code 0
10.244.4.245 - - [22/Aug/2024:10:01:09 +0000] "GET /metrics HTTP/1.1" 200 24626 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:01:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:01:20 +0000] "GET /metrics HTTP/1.1" 200 24627 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:01:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:01:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:01:39 +0000] "GET /metrics HTTP/1.1" 200 24639 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:01:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:01:50 +0000] "GET /metrics HTTP/1.1" 200 24625 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:01:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:02:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:02:09 +0000] "GET /metrics HTTP/1.1" 200 24627 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:02:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:02:20 +0000] "GET /metrics HTTP/1.1" 200 24641 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:02:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:02:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:02:39 +0000] "GET /metrics HTTP/1.1" 200 24640 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:02:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:02:50 +0000] "GET /metrics HTTP/1.1" 200 24627 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:02:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:03:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:03:09 +0000] "GET /metrics HTTP/1.1" 200 24628 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:03:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:03:20 +0000] "GET /metrics HTTP/1.1" 200 24626 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:03:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:03:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:03:39 +0000] "GET /metrics HTTP/1.1" 200 24635 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:03:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:03:50 +0000] "GET /metrics HTTP/1.1" 200 24627 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:03:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:04:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:04:09 +0000] "GET /metrics HTTP/1.1" 200 24627 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:04:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:04:20 +0000] "GET /metrics HTTP/1.1" 200 24641 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:04:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:04:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:04:39 +0000] "GET /metrics HTTP/1.1" 200 24625 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:04:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:04:50 +0000] "GET /metrics HTTP/1.1" 200 24627 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:04:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:05:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:05:09 +0000] "GET /metrics HTTP/1.1" 200 24627 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:05:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:05:20 +0000] "GET /metrics HTTP/1.1" 200 24625 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:05:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:05:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:05:39 +0000] "GET /metrics HTTP/1.1" 200 24625 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:05:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:05:50 +0000] "GET /metrics HTTP/1.1" 200 24625 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:05:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:06:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:06:09 +0000] "GET /metrics HTTP/1.1" 200 24625 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:06:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:06:20 +0000] "GET /metrics HTTP/1.1" 200 24639 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:06:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:06:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:06:39 +0000] "GET /metrics HTTP/1.1" 200 24626 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:06:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:06:50 +0000] "GET /metrics HTTP/1.1" 200 24645 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:06:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:07:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:07:09 +0000] "GET /metrics HTTP/1.1" 200 24628 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:07:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:07:20 +0000] "GET /metrics HTTP/1.1" 200 24630 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:07:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:07:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:07:39 +0000] "GET /metrics HTTP/1.1" 200 24625 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:07:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:07:50 +0000] "GET /metrics HTTP/1.1" 200 24644 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:07:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:08:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:08:09 +0000] "GET /metrics HTTP/1.1" 200 24650 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:08:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:08:20 +0000] "GET /metrics HTTP/1.1" 200 24649 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:08:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:08:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:08:39 +0000] "GET /metrics HTTP/1.1" 200 24658 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:08:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:08:50 +0000] "GET /metrics HTTP/1.1" 200 24662 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:08:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:09:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:09:09 +0000] "GET /metrics HTTP/1.1" 200 24648 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:09:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:09:20 +0000] "GET /metrics HTTP/1.1" 200 24685 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:09:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:09:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:09:39 +0000] "GET /metrics HTTP/1.1" 200 24736 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:09:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:09:50 +0000] "GET /metrics HTTP/1.1" 200 24721 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:09:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:10:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:10:09 +0000] "GET /metrics HTTP/1.1" 200 24736 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:10:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:10:20 +0000] "GET /metrics HTTP/1.1" 200 24723 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:10:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:10:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:10:39 +0000] "GET /metrics HTTP/1.1" 200 24723 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:10:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:10:50 +0000] "GET /metrics HTTP/1.1" 200 24736 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:10:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:11:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:11:09 +0000] "GET /metrics HTTP/1.1" 200 24721 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:11:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:11:20 +0000] "GET /metrics HTTP/1.1" 200 24738 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:11:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:11:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:11:39 +0000] "GET /metrics HTTP/1.1" 200 24736 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:11:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:11:50 +0000] "GET /metrics HTTP/1.1" 200 24736 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:11:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:12:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:12:09 +0000] "GET /metrics HTTP/1.1" 200 24724 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:12:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:12:20 +0000] "GET /metrics HTTP/1.1" 200 24725 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:12:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:12:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:12:39 +0000] "GET /metrics HTTP/1.1" 200 24737 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:12:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:12:50 +0000] "GET /metrics HTTP/1.1" 200 24735 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:12:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:13:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:13:09 +0000] "GET /metrics HTTP/1.1" 200 24737 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:13:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:13:20 +0000] "GET /metrics HTTP/1.1" 200 24722 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:13:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:13:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:13:39 +0000] "GET /metrics HTTP/1.1" 200 24723 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:13:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:13:50 +0000] "GET /metrics HTTP/1.1" 200 24736 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:13:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:14:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:14:09 +0000] "GET /metrics HTTP/1.1" 200 24724 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:14:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:14:20 +0000] "GET /metrics HTTP/1.1" 200 24737 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:14:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:14:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:14:39 +0000] "GET /metrics HTTP/1.1" 200 24738 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:14:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:14:50 +0000] "GET /metrics HTTP/1.1" 200 24724 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:14:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:15:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:15:09 +0000] "GET /metrics HTTP/1.1" 200 24722 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:15:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:15:20 +0000] "GET /metrics HTTP/1.1" 200 24722 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:15:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:15:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:15:39 +0000] "GET /metrics HTTP/1.1" 200 24725 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:15:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:15:50 +0000] "GET /metrics HTTP/1.1" 200 24723 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:15:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:16:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:16:09 +0000] "GET /metrics HTTP/1.1" 200 24723 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:16:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:16:20 +0000] "GET /metrics HTTP/1.1" 200 24736 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:16:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:16:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:16:39 +0000] "GET /metrics HTTP/1.1" 200 24723 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:16:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:16:50 +0000] "GET /metrics HTTP/1.1" 200 24737 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:16:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:17:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:17:09 +0000] "GET /metrics HTTP/1.1" 200 24724 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:17:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:17:20 +0000] "GET /metrics HTTP/1.1" 200 24724 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:17:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:17:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:17:39 +0000] "GET /metrics HTTP/1.1" 200 24722 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:17:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:17:50 +0000] "GET /metrics HTTP/1.1" 200 24721 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:17:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:18:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:18:09 +0000] "GET /metrics HTTP/1.1" 200 24722 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:18:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:18:20 +0000] "GET /metrics HTTP/1.1" 200 24722 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:18:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:18:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:18:39 +0000] "GET /metrics HTTP/1.1" 200 24725 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:18:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:18:50 +0000] "GET /metrics HTTP/1.1" 200 24723 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:18:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:19:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:19:09 +0000] "GET /metrics HTTP/1.1" 200 24722 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:19:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:19:20 +0000] "GET /metrics HTTP/1.1" 200 24722 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:19:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:19:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:19:39 +0000] "GET /metrics HTTP/1.1" 200 24722 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:19:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:19:50 +0000] "GET /metrics HTTP/1.1" 200 24723 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:19:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:20:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:20:09 +0000] "GET /metrics HTTP/1.1" 200 24722 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:20:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:20:20 +0000] "GET /metrics HTTP/1.1" 200 24723 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:20:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:20:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:20:39 +0000] "GET /metrics HTTP/1.1" 200 24724 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:20:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:20:50 +0000] "GET /metrics HTTP/1.1" 200 24724 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:20:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:21:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:21:09 +0000] "GET /metrics HTTP/1.1" 200 24722 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:21:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:21:20 +0000] "GET /metrics HTTP/1.1" 200 24725 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:21:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:21:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:21:39 +0000] "GET /metrics HTTP/1.1" 200 24744 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:21:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:21:50 +0000] "GET /metrics HTTP/1.1" 200 24723 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:21:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:22:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:22:09 +0000] "GET /metrics HTTP/1.1" 200 24724 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:22:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:22:20 +0000] "GET /metrics HTTP/1.1" 200 24745 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:22:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:22:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:22:39 +0000] "GET /metrics HTTP/1.1" 200 24721 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:22:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:22:50 +0000] "GET /metrics HTTP/1.1" 200 24751 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:22:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:23:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:23:09 +0000] "GET /metrics HTTP/1.1" 200 24750 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:23:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
2024/08/22 10:23:20 [info] 224#224 "netbox" application started
10.244.2.85 - - [22/Aug/2024:10:23:20 +0000] "GET /metrics HTTP/1.1" 200 24742 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:23:21 +0000] "GET / HTTP/1.1" 200 111016 "https://netbox.thdd.technik.kabel-deutschland.de/dcim/device-types/?manufacturer_id=10" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0"
10.244.1.1 - - [22/Aug/2024:10:23:21 +0000] "GET /extras/changelog/?per_page=25&embedded=True HTTP/1.1" 200 23854 "https://netbox.thdd.technik.kabel-deutschland.de/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0"
10.244.1.1 - - [22/Aug/2024:10:23:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:23:25 +0000] "GET /dcim/device-types/ HTTP/1.1" 200 159874 "https://netbox.thdd.technik.kabel-deutschland.de/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0"
10.244.1.1 - - [22/Aug/2024:10:23:29 +0000] "GET /dcim/device-types/?page=2 HTTP/1.1" 200 38313 "https://netbox.thdd.technik.kabel-deutschland.de/dcim/device-types/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0"
10.244.1.1 - - [22/Aug/2024:10:23:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:23:39 +0000] "GET /metrics HTTP/1.1" 200 33053 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:23:50 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:23:50 +0000] "GET /metrics HTTP/1.1" 200 21830 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:23:52 +0000] "POST /dcim/device-types/delete/ HTTP/1.1" 200 135019 "https://netbox.thdd.technik.kabel-deutschland.de/dcim/device-types/?page=2" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0"
10.244.1.1 - - [22/Aug/2024:10:23:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:23:55 +0000] "POST /dcim/device-types/delete/ HTTP/1.1" 302 0 "https://netbox.thdd.technik.kabel-deutschland.de/dcim/device-types/delete/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0"
10.244.1.1 - - [22/Aug/2024:10:23:55 +0000] "GET /dcim/device-types/ HTTP/1.1" 200 160096 "https://netbox.thdd.technik.kabel-deutschland.de/dcim/device-types/delete/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0"
10.244.1.1 - - [22/Aug/2024:10:24:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:24:09 +0000] "GET /metrics HTTP/1.1" 200 24745 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:24:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:24:20 +0000] "GET /metrics HTTP/1.1" 200 36856 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:24:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:24:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:24:39 +0000] "GET /metrics HTTP/1.1" 200 36858 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:24:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:24:50 +0000] "GET /metrics HTTP/1.1" 200 27483 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:24:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:25:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:25:09 +0000] "GET /metrics HTTP/1.1" 200 24745 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:25:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:25:20 +0000] "GET /metrics HTTP/1.1" 200 27481 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:25:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:25:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:25:39 +0000] "GET /metrics HTTP/1.1" 200 36858 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:25:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:25:50 +0000] "GET /metrics HTTP/1.1" 200 36860 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:25:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:26:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:26:09 +0000] "GET /metrics HTTP/1.1" 200 24744 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:26:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:26:20 +0000] "GET /metrics HTTP/1.1" 200 24747 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:26:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:26:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:26:39 +0000] "GET /metrics HTTP/1.1" 200 27540 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:26:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:26:50 +0000] "GET /metrics HTTP/1.1" 200 24759 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:26:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:27:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:27:09 +0000] "GET /metrics HTTP/1.1" 200 36857 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:27:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:27:20 +0000] "GET /metrics HTTP/1.1" 200 36858 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:27:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:27:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:27:39 +0000] "GET /metrics HTTP/1.1" 200 36858 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:27:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:27:50 +0000] "GET /metrics HTTP/1.1" 200 24745 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:27:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:28:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:28:09 +0000] "GET /metrics HTTP/1.1" 200 27554 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:28:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:28:20 +0000] "GET /metrics HTTP/1.1" 200 27554 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:28:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:28:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:28:39 +0000] "GET /metrics HTTP/1.1" 200 36855 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:28:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:28:50 +0000] "GET /metrics HTTP/1.1" 200 24745 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:28:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:29:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:29:09 +0000] "GET /metrics HTTP/1.1" 200 36858 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:29:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:29:20 +0000] "GET /metrics HTTP/1.1" 200 36860 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:29:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:29:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:29:39 +0000] "GET /metrics HTTP/1.1" 200 27570 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:29:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:29:50 +0000] "GET /metrics HTTP/1.1" 200 27577 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:29:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:30:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:30:09 +0000] "GET /metrics HTTP/1.1" 200 27576 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:30:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:30:20 +0000] "GET /metrics HTTP/1.1" 200 24747 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:30:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:30:34 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:30:39 +0000] "GET /metrics HTTP/1.1" 200 24748 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:30:44 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:30:50 +0000] "GET /metrics HTTP/1.1" 200 24745 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:30:54 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.1.1 - - [22/Aug/2024:10:31:04 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.4.245 - - [22/Aug/2024:10:31:09 +0000] "GET /metrics HTTP/1.1" 200 27579 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:31:14 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"
10.244.2.85 - - [22/Aug/2024:10:31:20 +0000] "GET /metrics HTTP/1.1" 200 24746 "-" "Prometheus/2.51.2"
10.244.1.1 - - [22/Aug/2024:10:31:24 +0000] "GET /login/ HTTP/1.1" 200 3071 "-" "kube-probe/1.30"

Please see the last events: 

LAST SEEN               TYPE      REASON      OBJECT                         MESSAGE
45m (x2 over 45m)       Warning   BackOff     Pod/netbox-5d979d9dfb-cxrcw    Back-off restarting failed container netbox in pod netbox-5d979d9dfb-cxrcw_netbox(2bba5979-c38e-4ccb-9e02-c0e62365fe32)
45m (x8 over 3d1h)      Normal    Pulled      Pod/netbox-5d979d9dfb-cxrcw    Container image "ghcr.io/netbox-community/netbox:v4.0.9" already present on machine
45m (x8 over 3d1h)      Normal    Created     Pod/netbox-5d979d9dfb-cxrcw    Created container netbox
45m (x8 over 3d1h)      Normal    Started     Pod/netbox-5d979d9dfb-cxrcw    Started container netbox
44m (x51 over 3d1h)     Warning   Unhealthy   Pod/netbox-5d979d9dfb-cxrcw    Readiness probe failed: Get "http://10.244.1.6:8080/login/": dial tcp 10.244.1.6:8080: connect: connection refused
21m (x93 over 2d19h)    Warning   Unhealthy   Pod/netbox-2-dfbd65b46-ggs7d   Readiness probe failed: Get "http://10.244.1.50:8080/login/": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
5m34s (x24 over 3d1h)   Warning   Unhealthy   Pod/netbox-5d979d9dfb-cxrcw    Readiness probe failed: Get "http://10.244.1.6:8080/login/": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
LeoColomb commented 2 months ago

Thanks for the additional information. Please keep the discussion in the same issue next time, that helps us to follow up.

From the little I can see, the readiness probe has failed a few times, but it's hard to say if this is the reason for the pod failure. My guess is that the heavy load of the bulk request is slowing down too much the app, making it almost unresponsive. That is only a guess, and is located on the app side anyway. @RangerRick Any idea?

moppmans commented 2 months ago

Thanks Leo...unfortunatly it's no topic off the bulk jobs I think.

If running scripts in background to add a device and parallel setting values via Gui, this happens also. In addition that's the same sync script used for a server installation of netbox (3.x).

I thought it could be a problem with redis or psql timeout and parallel requests.

RangerRick commented 2 months ago

In theory, both Redis and PostgreSQL should be able to handle many requests just fine, but it does smell like some kind of resource restriction of some kind. Devilishly hard to debug though.

The thing that makes me wonder is that you essentially had the same setup before, but with 3.x, I wonder if there's anyone over in NetBox proper who has seen a change in performance between 3 and 4. A lot has changed under the covers, new major Django version, core model changes, etc.

moppmans commented 2 months ago

Thanks Rick.

In addtion, I also tried this behavior with a server installation of netbox 4.x and there it's also fine. So for me this is something with the pods...but I am not a kubernetes specialist :-)

I think I need to test this with external psql or redis instance as well...

RangerRick commented 2 months ago

Hrm, back to the drawing board then. Yeah, a test with external servers would be interesting.

moppmans commented 2 months ago

So we have a solution. Per default the resource preset for PODs were set to small. While monitoring PODs status it occured "OOMkilled". Hard to see because it's in no log.

We set the resource preset no to xlarge:

resourcesPreset: "xlarge"

Now we have a fast and reliable netbox service.

tbartschat commented 2 months ago

Thanks for your help, as @moppmans mentioned it was just a resource issue the pod itselfs.