Open hannanbeediwale opened 1 year ago
I'm facing same error for some pods in open5s ns:
[ec2-user@ip-10-1-0-111 x86-Architecture]$ kubectl get -n open5gs pods NAME READY STATUS RESTARTS AGE open5gs-hss-deployment-68b4dc778f-s9p95 0/1 Init:0/1 0 13m open5gs-mme-deployment-77c4b4f8c-4t7kk 0/1 Init:0/1 0 13m open5gs-nrf-deployment-54f9cd8b65-5dkhm 1/1 Running 0 13m open5gs-pcrf-deployment-64ff487f65-85dqx 0/1 Init:0/1 0 13m open5gs-sgwc-deployment-56944dcd88-ztz7r 0/1 Init:0/1 0 13m open5gs-sgwu-deployment-76d4f6779b-m5qzq 1/1 Running 0 13m open5gs-smf-deployment-7775846df8-wmf5k 0/1 Init:0/1 0 13m open5gs-upf-deployment-78bccbffdd-ztpzc 1/1 Running 0 13m open5gs-webui-78975d9746-q7d4v 1/1 Running 0 13m
kubectl logs: [ec2-user@ip-10-1-0-111 x86-Architecture]$ kubectl logs -n open5gs open5gs-mme-deployment-77c4b4f8c-4t7kk Defaulted container "mme" out of: mme, init-mme (init) Error from server: Get "https://10.1.2.231:10250/containerLogs/open5gs/open5gs-mme-deployment-77c4b4f8c-4t7kk/mme": remote error: tls: internal error
Same error message for kube-system NS multus pods:
[ec2-user@ip-10-1-0-111 x86-Architecture]$ kubectl get -n kube-system pods NAME READY STATUS RESTARTS AGE aws-node-kr87b 2/2 Running 0 109m aws-node-s75l7 2/2 Running 0 109m coredns-58488c5db-hg956 1/1 Running 0 118m coredns-58488c5db-mv9vj 1/1 Running 0 118m kube-proxy-dll75 1/1 Running 0 109m kube-proxy-rn2wk 1/1 Running 0 109m multus-route53-service-operator-7bd78b8d7c-cp54s 0/1 CrashLoopBackOff 6 (98s ago) 7m17s multus-secondary-ip-ec2-mapping-operator-75c68fb48c-tczb9 0/1 CrashLoopBackOff 6 (85s ago) 7m17s [ec2-user@ip-10-1-0-111 x86-Architecture]$ kubectl logs -n kube-system multus-route53-service-operator-7bd78b8d7c-cp54s Error from server: Get "https://10.1.2.158:10250/containerLogs/kube-system/multus-route53-service-operator-7bd78b8d7c-cp54s/shell-operator": remote error: tls: internal error [ec2-user@ip-10-1-0-111 x86-Architecture]$
Error is observed after executing Staging environment- step 2 - "Run the ./cluster_initializer.sh"
Hi
Facing issue in the last step at Helm Deployment:
Install Helm chart with the following command: helm -n open5gs install -f values.yaml epc-core ./
[[ec2-user@ip-10-0-0-14 x86-Architecture]$ kubectl get -n open5gs pods NAME READY STATUS RESTARTS AGE open5gs-hss-deployment-5d856c4f7b-jgmfx 0/1 Init:0/1 0 19m open5gs-mme-deployment-848cd7766-5t82q 0/1 Init:0/1 0 19m open5gs-nrf-deployment-55cc9fc696-zczwz 1/1 Running 0 19m open5gs-pcrf-deployment-f8fb984fb-7w7d8 0/1 Init:0/1 0 19m open5gs-sgwc-deployment-5bb6d97475-9p6st 0/1 Init:0/1 0 19m open5gs-sgwu-deployment-bfcc6c8cf-c98rr 1/1 Running 0 19m open5gs-smf-deployment-7f679b945-x79l4 0/1 Init:0/1 0 19m open5gs-upf-deployment-64cf6b6499-wpzrz 1/1 Running 0 19m open5gs-webui-5dd5944c56-jc59c 1/1 Running 0 19m [ec2-user@ip-10-0-0-14 x86-Architecture]$ [ec2-user@ip-10-0-0-14 x86-Architecture]$
Completed all the install steps Following this link to install:https://aws.amazon.com/blogs/opensource/open-source-mobile-core-network-implementation-on-amazon-elastic-kubernetes-service/
[ec2-user@ip-10-0-0-218 x86-Architecture]$ kubectl -n open5gs get secret NAME TYPE DATA AGE diameter-ca Opaque 1 8h hss-tls kubernetes.io/tls 2 8h mme-tls kubernetes.io/tls 2 8h mongodb-ca Opaque 1 8h pcrf-tls kubernetes.io/tls 2 8h smf-tls kubernetes.io/tls 2 8h
[ec2-user@ip-10-0-0-218 x86-Architecture]$ kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE aws-node-r5bsr 1/1 Running 0 9m6s coredns-594b88dc6c-njnw9 1/1 Running 0 19m coredns-594b88dc6c-pj24l 1/1 Running 0 19m kube-multus-ds-5nqt9 1/1 Running 0 9m6s kube-proxy-cdbj4 1/1 Running 0 9m6s multus-route53-service-operator-645c94c4bb-5c4wf 1/1 Running 0 19m multus-secondary-ip-ec2-mapping-operator-74586574d4-zrzwz 1/1 Running 0 19m
[ec2-user@ip-10-0-0-14 x86-Architecture]$ kubectl describe pod open5gs-smf-deployment-7f679b945-x79l4 --namespace open5gs Name: open5gs-smf-deployment-7f679b945-x79l4 Namespace: open5gs Priority: 0 Node: ip-10-0-2-39.ca-central-1.compute.internal/10.0.2.39 Start Time: Mon, 22 May 2023 20:01:00 +0000 Labels: epc-mode=smf pod-template-hash=7f679b945 Annotations: k8s.v1.cni.cncf.io/network-status: [{ "name": "aws-cni", "interface": "dummybb591efea73", "ips": [ "10.0.2.228" ], "mac": "0", "default": true, "dns": {} },{ "name": "open5gs/ipvlan-multus-sub-1-cp", "interface": "net1", "ips": [ "10.0.4.208" ], "mac": "02:13:f3:9b:f0:3c", "dns": {} }] k8s.v1.cni.cncf.io/networks: [ { "name": "ipvlan-multus-sub-1-cp", "interface": "net1" } ] k8s.v1.cni.cncf.io/networks-status: [{ "name": "aws-cni", "interface": "dummybb591efea73", "ips": [ "10.0.2.228" ], "mac": "0", "default": true, "dns": {} },{ "name": "open5gs/ipvlan-multus-sub-1-cp", "interface": "net1", "ips": [ "10.0.4.208" ], "mac": "02:13:f3:9b:f0:3c", "dns": {} }] route53-service-name: [ { "name": "s5.smf.open5gs.service", "multus-int": "ipvlan-multus-sub-1-cp" } ] Status: Pending IP: 10.0.2.228 IPs: IP: 10.0.2.228 Controlled By: ReplicaSet/open5gs-smf-deployment-7f679b945 Init Containers: init-smf: Container ID: docker://349bab56b9e0e28c70594489ae364e8d96d504c30d2c17611f070f39e986dc84 Image: busybox:1.28 Image ID: docker-pullable://busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 Port:
Host Port:
Command:
sh
-c
Args:
until nslookup gx.pcrf.open5gs.service >> /dev/null; do echo waiting for pcrf DNS record to be ready;done; until nslookup sx.upf.open5gs.service >> /dev/null; do echo waiting for upf DNS record to be ready; done
State: Running
Started: Mon, 22 May 2023 20:01:02 +0000
Ready: False
Restart Count: 0
Environment:
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-s4zq9 (ro)
Containers:
smf:
Container ID:
Host Port:
Command:
/bin/sh
-c
Args:
sleep 10; open5gs-smfd -c /open5gs/config-map/smf.yaml;
State: Waiting
Reason: PodInitializing
Ready: False
Restart Count: 0
Environment:
Mounts:
/open5gs/config-map/diameter-smf.conf from open5gs-smf-diameter (rw,path="diameter-smf.conf")
/open5gs/config-map/smf.yaml from open5gs-smf-config (rw,path="smf.yaml")
/open5gs/diameter-ca/ from diameter-ca (rw)
/open5gs/tls/ from smf-tls (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-s4zq9 (ro)
Conditions:
Type Status
Initialized False
Ready False
ContainersReady False
PodScheduled True
Volumes:
open5gs-smf-config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: open5gs-smf-config
Optional: false
open5gs-smf-diameter:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: open5gs-smf-diameter
Optional: false
diameter-ca:
Type: Secret (a volume populated by a Secret)
SecretName: diameter-ca
Optional: false
smf-tls:
Type: Secret (a volume populated by a Secret)
SecretName: smf-tls
Optional: false
kube-api-access-s4zq9:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional:
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: nodegroup=control-plane
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
Image: 523531615794.dkr.ecr.ca-central-1.amazonaws.com/hannanrepo:open5gs-x86-aio Image ID:
Port:
Normal Scheduled 6m4s default-scheduler Successfully assigned open5gs/open5gs-smf-deployment-7f679b945-x79l4 to ip-10-0-2-39.ca-central-1.compute.internal Normal AddedInterface 6m3s multus Add eth0 [10.0.2.228/32] from aws-cni Normal AddedInterface 6m3s multus Add net1 [10.0.4.208/24] from open5gs/ipvlan-multus-sub-1-cp Normal Pulled 6m3s kubelet Container image "busybox:1.28" already present on machine Normal Created 6m3s kubelet Created container init-smf Normal Started 6m2s kubelet Started container init-smf [ec2-user@ip-10-0-0-14 x86-Architecture]$