search

nephio-project / nephio

Nephio is a Kubernetes-based automation platform for deploying and managing highly distributed, interconnected workloads such as 5G Network Functions, and the underlying infrastructure on which those workloads depend.
Apache License 2.0
102 stars 53 forks source link

Nephio SDK: (PoC) --- Flux based Helm deployment of free5gc NFs #504

Open s3wong opened 7 months ago

s3wong commented 7 months ago

Verify use of Flux helm-controller to deploy all free5gc NFs via towards5gs Helm charts (even one by one). Publish the ConfigMap used as corresponding values.yaml

lapentad commented 6 months ago

Tested Orange TowardsFree5gc Helm charts 1.1.7 in a single kind cluster

  1. Installed management nephio cluster via ansible script 
    wget -O - https://raw.githubusercontent.com/nephio-project/test-infra/main/e2e/provision/init.sh | \
sudo NEPHIO_BRANCH=main \
 NEPHIO_USER=ubuntu \
 NEPHIO_E2E=false \
 bash
  2. Installed Multus And Flannel 
    kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl apply -f https://raw.githubusercontent.com/nephio-project/catalog/main/infra/capi/multus/multus-daemonset-thick.yml
  3. Added CNIs into the running cluster and created eth1
    • Downloaded cni binaries from https://github.com/containernetworking/plugins
    • Extracted them inside the docker container (kind-control-plane for the management cluster) in /opt/cni/bin/
    • Added eth1 network interface (for UPF) to the same cluster, as type dummy

One line command(long)

mkdir -p $HOME/tool && GET_VER=$(curl -L -s https://github.com/containernetworking/plugins/releases/latest | grep '^\s*v' | sed 's/ //g') && curl -Lo ./cni-plugins.tgz https://github.com/containernetworking/plugins/releases/download/$GET_VER/cni-plugins-linux-amd64-$GET_VER.tgz && sudo tar -zxvf cni-plugins.tgz --directory $HOME/tool/ && container_name="kind-control-plane" && docker_id=$(docker ps -q --filter "name=$container_name") && if [ -z "$docker_id" ]; then echo "Container '$container_name' is not running."; exit 1; else echo "Docker ID of '$container_name': $docker_id"; docker cp $HOME/tool/ $docker_id:/opt/cni/bin/; docker exec $docker_id cp /opt/cni/bin/tool/* /opt/cni/bin/; docker exec $docker_id rm -rf /opt/cni/bin/tool/; docker exec $docker_id ip link add eth1 type dummy; docker exec $docker_id ip addr add 1.1.1.1/24 dev eth1; docker exec $docker_id ip link set eth1 up; fi

  1. Made sure Mongodb had a persistent volume on the machine

    mkdir $HOME/kubedata

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: PersistentVolume
metadata:
name: example-local-pv9
labels:
project: free5gc
spec:
capacity:
storage: 8Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
local:
path: /home/$USER/kubedata
nodeAffinity:
required:
  nodeSelectorTerms:
  - matchExpressions:
    - key: kubernetes.io/hostname
      operator: In
      values:
      - kind-control-plane
EOF

  2. Deployed Flux onto mgmt cluster using https://github.com/nephio-project/catalog/tree/main/nephio/optional/flux-helm-controllers

  3. Deployed custom package onto mgmt from https://github.com/Nordix/nordix-nephio-packages/tree/free5gc-flux-test/packages/free5gc-flux

  4. Resources come up on namespace demo

    
Every 2.0s: kubectl get po -n demo                                                         vm-nephio-e2e-test: Tue Feb 20 15:58:30 2024

NAME READY STATUS RESTARTS AGE demo-free5gc-free5gc-amf-amf-68c7b6d7b9-pfth5 1/1 Running 0 83s demo-free5gc-free5gc-ausf-ausf-6ff9d98c96-j2kb7 1/1 Running 0 83s demo-free5gc-free5gc-dbpython-dbpython-6c496ff5dd-9vhr8 1/1 Running 0 83s demo-free5gc-free5gc-nrf-nrf-6677f95c49-2fnjz 1/1 Running 0 83s demo-free5gc-free5gc-nssf-nssf-5b999d85dc-j82n9 1/1 Running 0 83s demo-free5gc-free5gc-pcf-pcf-7476987549-pw64f 1/1 Running 0 83s demo-free5gc-free5gc-smf-smf-6cd5799c45-tnw5b 1/1 Running 0 83s demo-free5gc-free5gc-udm-udm-7bcf58d499-4mp5r 1/1 Running 0 83s demo-free5gc-free5gc-udr-udr-76c4f4467-gsnf2 1/1 Running 0 83s demo-free5gc-free5gc-upf-upf-6c684879b8-w7j55 1/1 Running 0 83s demo-free5gc-free5gc-webui-webui-8c7f4576f-sf4vf 1/1 Running 0 83s mongodb-0 1/1 Running 0 83s

ubuntu@vm-nephio-e2e-test:~$ k get po -A NAMESPACE NAME READY STATUS RESTARTS AGE backend-system resource-backend-controller-678558fbfb-5qxn6 2/2 Running 0 160m capd-system capd-controller-manager-7f6b64fbcb-mpz64 1/1 Running 0 159m capi-kubeadm-bootstrap-system capi-kubeadm-bootstrap-controller-manager-55d5767547-l46f5 1/1 Running 0 159m capi-kubeadm-control-plane-system capi-kubeadm-control-plane-controller-manager-757b4c596f-nm6m2 1/1 Running 0 159m capi-system capi-controller-manager-69678b75f9-dlkvl 1/1 Running 0 159m cert-manager cert-manager-6954d7bbbf-kqq8s 1/1 Running 0 160m cert-manager cert-manager-cainjector-84bdff4846-q4tgq 1/1 Running 0 160m cert-manager cert-manager-webhook-85b6b76d9b-8ff2b 1/1 Running 0 160m config-management-monitoring otel-collector-798c8784bd-82j25 1/1 Running 0 156m config-management-system config-management-operator-6946b77565-r5ph8 1/1 Running 0 156m config-management-system reconciler-manager-5b5d8557-qqjxg 2/2 Running 0 156m config-management-system root-reconciler-mgmt-6fdf94dfd4-smnmw 4/4 Running 0 154m demo demo-free5gc-free5gc-amf-amf-68c7b6d7b9-pfth5 1/1 Running 0 26m demo demo-free5gc-free5gc-ausf-ausf-6ff9d98c96-j2kb7 1/1 Running 0 26m demo demo-free5gc-free5gc-dbpython-dbpython-6c496ff5dd-9vhr8 1/1 Running 0 26m demo demo-free5gc-free5gc-nrf-nrf-6677f95c49-2fnjz 1/1 Running 0 26m demo demo-free5gc-free5gc-nssf-nssf-5b999d85dc-j82n9 1/1 Running 0 26m demo demo-free5gc-free5gc-pcf-pcf-7476987549-pw64f 1/1 Running 0 26m demo demo-free5gc-free5gc-smf-smf-6cd5799c45-tnw5b 1/1 Running 0 26m demo demo-free5gc-free5gc-udm-udm-7bcf58d499-4mp5r 1/1 Running 0 26m demo demo-free5gc-free5gc-udr-udr-76c4f4467-gsnf2 1/1 Running 0 26m demo demo-free5gc-free5gc-upf-upf-6c684879b8-w7j55 1/1 Running 0 26m demo demo-free5gc-free5gc-webui-webui-8c7f4576f-sf4vf 1/1 Running 0 26m demo mongodb-0 1/1 Running 0 26m flux-system helm-controller-5895785fbc-4r9gr 1/1 Running 0 29m flux-system source-controller-7556cdbd66-fjp6c 1/1 Running 0 29m gitea gitea-0 1/1 Running 0 160m gitea gitea-memcached-6777864fbd-7pslj 1/1 Running 0 160m gitea gitea-postgresql-0 1/1 Running 0 160m kube-flannel kube-flannel-ds-5l4l8 1/1 Running 0 30m kube-system coredns-5d78c9869d-65k8f 1/1 Running 0 161m kube-system coredns-5d78c9869d-sg2hh 1/1 Running 0 161m kube-system etcd-kind-control-plane 1/1 Running 0 161m kube-system kindnet-6znvb 1/1 Running 0 161m kube-system kube-apiserver-kind-control-plane 1/1 Running 0 161m kube-system kube-controller-manager-kind-control-plane 1/1 Running 0 161m kube-system kube-multus-ds-ssgh2 1/1 Running 0 30m kube-system kube-proxy-tdd2f 1/1 Running 0 161m kube-system kube-scheduler-kind-control-plane 1/1 Running 0 161m local-path-storage local-path-provisioner-6bc4bddd6b-fbbg7 1/1 Running 0 161m metallb-system controller-7948676b95-hn4kb 1/1 Running 0 160m metallb-system speaker-z2nmc 1/1 Running 0 160m nephio-system nephio-controller-f476cdc64-kkd6g 2/2 Running 0 157m nephio-system token-controller-86c699bfd-299xx 2/2 Running 0 157m nephio-webui nephio-webui-66cccf647f-pqdjw 1/1 Running 0 155m network-config network-config-controller-5474d8fdf4-5xb8b 2/2 Running 0 156m porch-system function-runner-bc9f45498-fhnkd 1/1 Running 0 158m porch-system function-runner-bc9f45498-qbz49 1/1 Running 0 158m porch-system porch-controllers-6ffff96c8d-hhskt 1/1 Running 0 158m porch-system porch-server-6f87887f68-qd2c9 1/1 Running 0 158m resource-group-system resource-group-controller-manager-6c9d56d88-t4npg 3/3 Running 0 156m

lapentad commented 6 months ago

@s3wong This is the configmap used to deploy the free5gc NFs

https://github.com/Nordix/nordix-nephio-packages/blob/free5gc-flux-test/packages/free5gc-flux/free5gc-cm.yaml

s3wong commented 6 months ago

@lapentad Thanks