search

nephio-project / nephio

Nephio is a Kubernetes-based automation platform for deploying and managing highly distributed, interconnected workloads such as 5G Network Functions, and the underlying infrastructure on which those workloads depend.
Apache License 2.0
101 stars 53 forks source link

E2E Testing : Free5gc NF packages are deployed successfully onto workload clusters using Nephio. #238

Closed gvbalaji closed 1 year ago

gvbalaji commented 1 year ago

Make sure all PODs are up and running Make sure ping works between SMF and UPF

johnbelamaric commented 1 year ago

Probably also need packages to represent the actual function instances, including all their requirement CRs, etc., that go through the package specialization process.

/assign @vjayaramrh

Open issues:

vjayaramrh commented 1 year ago

@johnbelamaric @henderiw Should the value for the name field at https://github.com/nephio-project/free5gc-packages/blob/main/pkg-example-amf-bp/package-context.yaml#L8 be the name of the package i.e pkg-example-amf-bp instead of just example cc: @rravindran123

johnbelamaric commented 1 year ago

That gets filled in automatically by kpt/Porch when the package published to a deployment repository (by Porch) or pulled with kpt get --for-deployment.

vjayaramrh commented 1 year ago

Where are the instructions to deploy the free5gc nf packages? I am looking for instructions something similar to step 5 at https://github.com/nephio-project/test-infra/blob/main/e2e/provision/README.md

johnbelamaric commented 1 year ago

https://github.com/nephio-project/test-infra/pull/61

I haven't run through and tested all this, but this is quite close to the flow we will have overall.

vjayaramrh commented 1 year ago

Update on June 5th:

Need a fix for this to continue further testing to close out the issue. Is there an issue already open for this? If not, I can open one.

@johnbelamaric , Meanwhile, Who can provide the instructions to setup the SMF and UPF so that the ping between them can be tested out. Want to get familar with those instructions until the above pod issues are debugged and resolved?

johnbelamaric commented 1 year ago

https://github.com/nephio-project/nephio/issues/256

I think @henderiw and the package specialization team are working on the SMF/UPF/AMF integration.

henderiw commented 1 year ago

Yes open PR(s) Are:

259 -> this is a big one since we moved from clusterCtx to workload cluster. I updated all the test files. So many files, but this is the main change

https://github.com/nephio-project/api/pull/38 -> this allows to validate empty as not ready. Right now I use empty string.

https://github.com/nephio-project/api/pull/39 -> this allows for ipv6 and ipv4. So this allows us to specify what we want rather than hardcoding. It is a small change I want to incorporate after #259 is approved

henderiw commented 1 year ago

so I have the specialization running, but to continue the PR chain I need the above 3 approved to proceed.

apiVersion: workload.nephio.org/v1alpha1

  | kind: UPFDeployment   | metadata: # kpt-merge: /upf-   | name: "upf-cluster03"   | annotations:   | internal.kpt.dev/upstream-identifier: 'workload.nephio.org|UPFDeployment|default|upf-'   | spec:   | capacity:   | maxDownlinkThroughput: 10G   | maxUplinkThroughput: 10G   | interfaces:   | - name: n3   | ipv4:   | address: 10.0.0.10/24   | gateway: 10.0.0.1   | vlanID: 10   | - name: n4   | ipv4:   | address: 10.0.0.10/24   | gateway: 10.0.0.1   | vlanID: 10   | - name: n6   | ipv4:   | address: 10.0.0.10/24   | gateway: 10.0.0.1   | vlanID: 10   | networkInstances:   | - name: vpc-internal   | interfaces:   | - n4   | - name: vpc-internet   | dataNetworks:   | - name: vpc-internet   | pool:   | - prefix: 172.0.0.0/8   | interfaces:   | - n6   | - name: vpc-ran   | interfaces:   | - n3

apiVersion: workload.nephio.org/v1alpha1

  | kind: UPFDeployment   | metadata: # kpt-merge: /upf-   | name: "upf-cluster03"   | annotations:   | internal.kpt.dev/upstream-identifier: 'workload.nephio.org|UPFDeployment|default|upf-'   | spec:   | capacity:   | maxDownlinkThroughput: 10G   | maxUplinkThroughput: 10G   | interfaces:   | - name: n3   | ipv4:   | address: 10.0.0.10/24   | gateway: 10.0.0.1   | vlanID: 10   | - name: n4   | ipv4:   | address: 10.0.0.10/24   | gateway: 10.0.0.1   | vlanID: 10   | - name: n6   | ipv4:   | address: 10.0.0.10/24   | gateway: 10.0.0.1   | vlanID: 10   | networkInstances:   | - name: vpc-internal   | interfaces:   | - n4   | - name: vpc-internet   | dataNetworks:   | - name: vpc-internet   | pool:   | - prefix: 172.0.0.0/8   | interfaces:   | - n6   | - name: vpc-ran   | interfaces:   | - n3

apiVersion: workload.nephio.org/v1alpha1

  | kind: UPFDeployment   | metadata: # kpt-merge: /upf-   | name: "upf-cluster03"   | annotations:   | internal.kpt.dev/upstream-identifier: 'workload.nephio.org|UPFDeployment|default|upf-'   | spec:   | capacity:   | maxDownlinkThroughput: 10G   | maxUplinkThroughput: 10G   | interfaces:   | - name: n3   | ipv4:   | address: 10.0.0.10/24   | gateway: 10.0.0.1   | vlanID: 10   | - name: n4   | ipv4:   | address: 10.0.0.10/24   | gateway: 10.0.0.1   | vlanID: 10   | - name: n6   | ipv4:   | address: 10.0.0.10/24   | gateway: 10.0.0.1   | vlanID: 10   | networkInstances:   | - name: vpc-internal   | interfaces:   | - n4   | - name: vpc-internet   | dataNetworks:   | - name: vpc-internet   | pool:   | - prefix: 172.0.0.0/8   | interfaces:   | - n6   | - name: vpc-ran   | interfaces:   | - n3

henderiw commented 1 year ago

So the steps for me are:

  1. approve the above PR(s)
  2. do some further changes in the functions to incorporate the API(s)
  3. add the new packages as a consequence of step1/2
  4. upload images
vjayaramrh commented 1 year ago

I had to do a little hack for now based on https://github.com/nephio-project/nephio-example-packages/blob/main/cluster-capi-kind-docker-templates/docker_machine_template_worker.yaml#L12 that is mountable to make progress.

I had to create a directory /opt/cni/bin/edge01/mongodb and modify the PV yaml file as below to point to the newly created directory and apply the yaml file to create a PV for now

ubuntu@nephio-r1-e2e:~$ cat pv-edge01.yaml 
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-loc-sc
spec:
  persistentVolumeReclaimPolicy: Delete
    #storageClassName: local-path
  capacity:
    storage: 7Gi
  accessModes:
    - ReadWriteOnce
  local:
    path: "/opt/cni/bin/edge01/mongodb"
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/os
          operator: In
          values:
          - linux
vjayaramrh commented 1 year ago

June 6th Update: Followed instructions at https://github.com/nephio-project/test-infra/blob/main/e2e/provision/README.md and was able to execute steps till step 9 with workarounds. Step 10 that attempts to deploys UPF does not deploy any pods. Need instructions to deploy SMF as well

gvbalaji commented 1 year ago

Depends on WIm's hydration work and operator work from Simon merged

johnbelamaric commented 1 year ago

So, we have free5gc-cp up and running. We have two more PRs to merge to get the operator up and running. And I think Wim nearly has the hydration/specialization work done.

gvbalaji commented 1 year ago

Sequence of operations :

johnbelamaric commented 1 year ago

This is working now. There is still some flakiness, I think due to concurrency issues. But overall it does work.