smart-edge-open / converged-edge-experience-kits

Source code for experience kits with Ansible-based deployment.
Apache License 2.0
37 stars 40 forks source link

Error while deploying Single-node Network Edge cluster #39

Closed amitinfo2k closed 4 years ago

amitinfo2k commented 4 years ago

Hello,

I am trying to deploy a single-node Network Edge cluster (using command ./deploy_ne.sh -f minimal single). It's failing while waiting for Grafana pod.

TASK [telemetry/grafana : wait for Grafana pod to be ready] ****************************************************************************
task path: /home/centos/openness-experience-kits/roles/telemetry/grafana/tasks/main.yml:205
fatal: [controller]: FAILED! => {
    "changed": false,
    "cmd": [
        "kubectl",
        "wait",
        "--selector=app.kubernetes.io/instance=grafana",
        "--namespace=telemetry",
        "--for=condition=Ready",
        "pods",
        "--timeout=600s"
    ],
    "delta": "0:10:00.283865",
    "end": "2020-07-22 14:22:32.815846",
    "rc": 1,
    "start": "2020-07-22 14:12:32.531981"
}

STDERR:
error: timed out waiting for the condition on pods/grafana-6b79c984b-tmqk4

MSG:
non-zero return code

Following pods are in error states.


...
openness      eaa-6f8b94c9d7-jcp9v                                        0/1     ErrImageNeverPull        0          50m
openness      edgedns-kl4qh                                               0/1     ErrImageNeverPull        0          50m
openness      interfaceservice-zcgbb                                      0/1     ErrImageNeverPull        0          45m
..
telemetry     grafana-6b79c984b-tmqk4                                     0/2     Init:RunContainerError   0          35m
...

For Graphana pod I see following error:


Normal   Created     30m (x2 over 32m)    kubelet, openness  Created container grafana-sc-datasources
Warning  Failed      28m (x2 over 30m)    kubelet, openness  Error: context deadline exceeded
Warning  FailedSync  5m48s (x9 over 26m)  kubelet, openness  error determining status: rpc error: code = DeadlineExceeded desc = context deadline exceeded

tomaszwesolowski commented 4 years ago

Hi, Could you provide output from kubectl describe pod grafana_pod_name command?

amitinfo2k commented 4 years ago

I don't have the complete o/p now, but the snippet is below:

Normal   Created     30m (x2 over 32m)    kubelet, openness  Created container grafana-sc-datasources
Warning  Failed      28m (x2 over 30m)    kubelet, openness  Error: context deadline exceeded
Warning  FailedSync  5m48s (x9 over 26m)  kubelet, openness  error determining status: rpc error: code = DeadlineExceeded desc = context deadline exceeded
tomaszwesolowski commented 4 years ago

Well, with this limited output there's not much we can do to help you. Maybe try deploy it one more time and check if error occurs again, if it does please provide output from following commands: kubectl describe po grafana-pod_name -n telemetry kubectl logs grafana-pod_name -n telemetry - c grafana-sc-datasources