Create a single-node bridged network version of the GoEPC use case using Multus

[taylor]

Create a single-node bridged network version of the GoEPC use case using Multus

This is a POC step in reaching the goal of a multi-node segmented and routed user plane

Expected results for this POC:

• Can send traffic through the bridged networks connecting ENB, S-GW, P-GW and the end point test server

Tasks:

• [x] Create Docker images for the CNFs to run on the K8s deployed to Packet

  • [x] Containerized P-GW (#313)
  • [x] Containerized S-GW (#314)
  • [x] Containerized Pseudo ENB (#315 )
  • [x] Containerized Pseudo MME (#316)

• [x] Create Helm charts for CNFs

• [x] Add initial multus support to k8s-infra

• [x] Update provisioning script to support provisioning K8s with Multus

• [x] Create host bridges for GoEPC through Kubernetes/Multus

  • [x] Create NetworkAttachmentDefinition for bridges (YAML)
  • [x] Verify that bridges are created by Multus

• [x] Create Helm charts for deploying user-plane components

  • [x] Add GoGTP configuration through ConfigMap

• [x] Prepare endpoints for verifying traffic through EPC

  • [x] Server pod is running basic webserver (Python)
  • [x] Client has ping and wget pre-installed for easy testing

• [x] Add initial README with steps needed to deploy and verify

• [x] Add testing and acceptance criteria for peer review

POC release tasks:

• [x] Rename folders under https://github.com/cncf/cnf-testbed/tree/go-gtp/examples/use_case/gogtp-k8s to match the milestones
  • [x] k8s_routable => k8s_flat_network
  • [x] K8s_multus => k8s_bridged_networks
  • [x] Update this ticket to match the new folder names
• [x] Update k8s bridged network README to match new folder name
• [x] If needed, update code to match folder name change
• [x] Update README to remove multus temp fix requirements
• [x] Add instructions to README to say how to use the correct k8s-infra correctly
• [ ] Merge the current go-gtp branch into master
• [ ] Create a tag on master called k8s_bridged_network-v0.1 (including go-gtp and deployment tools)

Testing:

• [ ] QA testing and peer review following acceptance criteria https://github.com/cncf/cnf-testbed/issues/327#issuecomment-582148592
• [ ] Update/add documentation, as needed for missing items
  • [ ] Common GoEPC K8s documentation
  • [ ] Bridged multus use case documentation

---

[taylor]

Current status of GoGTP K8s Multus implementation

• Multus deployed using Kubespray
  • Currently through modified image: crosscloudci/k8s-infra:multus
  • A better solution will be made (Michael/Denver)
• Multus configuration must be updated on worker after deploying
  • Issue with Multus and Calico (CNI version issue)
  • Steps documented in README
  • A better solution in progress (Denver/Michael)
• Uses Bridge CNI to create networks between GoGTP applications
  • Results in point-to-point connections as shown in diagrams
  • Networks created using Kubectl and provided yaml file
• Deploys GoGTP EPC plus endpoints (UE and External Server)
  • Everything handled through Helm
  • Can be tested using HTTP and ICMP from "UE" POD
• GoGTP EPC componenets are run as privileged
  • IPv4 forwarding disabled when only using NET_ADMIN capability
  • Privileged is needed to allow forwarding (sysctl)
  • Endpoints are run with NET_ADMIN capability (might not be needed)
• Steps to configure, deploy and test included in README file
• Uses images from examples/network_functions/go-gtp
  • Plus Ubuntu:18.04 for endpoints

Initial commits:

• Add temp Multus support https://github.com/cncf/cnf-testbed/commit/681e5d1e586045b9dfd7840165ff98137f5a8478
• Add GoGTP Multus use-case https://github.com/cncf/cnf-testbed/commit/c5ed92fe62d341b1f9fb7cb0c3b82ce038031392
• Create Kubespray patches for supporting calico, cni, & multus https://github.com/crosscloudci/k8s-infra/commit/56ac7a9f7d9c2a27e9658d27e2d0b3226515069e
• Enable calico, cni, & multus https://github.com/crosscloudci/k8s-infra/commit/4f64477fa8742e622bf07b1f0bdb82bed6675e15
• Patch calico & enable cni https://github.com/crosscloudci/k8s-infra/commit/525b1da4cf11f86240974a356ab43261d45234d4

[michaelspedersen]

• Use-case is available in CNF Testbed (go-gtp branch)

  • Dir: [examples/use_case/gogtp-k8s/k8s_bridged_networks](https://github.com/cncf/cnf-testbed/tree/go-gtp/examples/use_case/gogtp-k8s/k8s_bridged_networks)

• K8s-infra image available as crosscloudci/k8s-infra:multus

• Provisioning script has been updated to support Multus

  • See: tools/kubernetes_provisioning.sh (go-gtp branch)

• Bridge configuration using Multus available in use-case

  • Deploy: kubectl apply -f gogtp-bridges.yml
  • Verify: kubectl get network-attachment-definitions
  • Delete: kubectl delete -f gogtp-bridges.yml

• The use-case uses pre-built images available through Dockerhub

  • Local images can be built if needed
  • Dir: examples/network_functions/go-gtp (go-gtp branch)
  • Pulls from master of https://github.com/wmnsk/go-gtp

• Traffic can be verified from the gogtp-ue pod

  • wget http://10.0.1.201
  • ping 10.0.1.201

[taylor]

Acceptance Criteria:

Prereqs:

• Deploy K8s cluster with multus support; See https://github.com/cncf/cnf-testbed/tree/go-gtp/examples/use_case/gogtp-k8s/k8s_bridged_networks#prerequisites
• Clone the cnf-testbed
• Switch to the tag k8s_bridged_network-v0.1 (git checkout k8s_bridged_network-v0.1)

Multus single-node bridged network user plane is successfully deployed

• [ ] Go to the folder examples/use_case/gogtp-k8s/k8s_bridged_networks
• [ ] Deploy the use case to K8s following the documentation
  • [ ] set environment variable for KUBECONFIG (replace path to match your location): export KUBECONFIG=<path>/<to>/kubeconfig
  • [ ] Configure the bridges to be used with Multus to connect the pods: kubectl apply -f gogtp-bridges.yml
  • [ ] Deploy the CNFs, external end-point test server, and testing system acting as "user equipment": helm install ./gogtp/
• [ ] Wait for approximately 30 seconds, then check that the eNB is running: kubectl logs gogtp-enb
• Expected results should look something like:

  [eNB] 2020/01/31 13:23:46 Established S1-MME connection with 172.21.1.12:36412
  [eNB] 2020/01/31 13:23:46 Started serving S1-U on 172.21.0.11:2152
  [eNB] 2020/01/31 13:23:46 Successfully established tunnel for 001010000000001

Test HTTP GET to the end-point server

• [ ] Deploy use case and verify the eNB is running
• [ ] Send traffic following https://github.com/cncf/cnf-testbed/tree/go-gtp/examples/use_case/gogtp-k8s/k8s_bridged_networks#testing-the-gogtp-service-chain
  • [ ] Get to the CLI on the test system: kubectl exec -it gogtp-ue /bin/bash
  • [ ] Test that HTTP requests work to the testing end-point server by running: wget http://10.0.1.201

Expected results: You should see a successful HTTP request (200 OK), followed by index.html being saved.

Test ICMP/ping to the end-point server

• [ ] Deploy use case and verify the eNB is running
• [ ] Send traffic following https://github.com/cncf/cnf-testbed/tree/go-gtp/examples/use_case/gogtp-k8s/k8s_bridged_networks#testing-the-gogtp-service-chain
  • [ ] Get to the CLI on the test system: kubectl exec -it gogtp-ue /bin/bash
  • [ ] Run: ping -c 5 10.0.1.201

Expected results:

• 5 packets transmitted, 5 received, 0% packet loss

[taylor]

@denverwilliams and @wavell if the tasks and acceptance criteria then go ahead and please try to follow the A/C and test. Update the docs as needed.