Chapter 6. Network Control

[ ] Make a Chapter6.md file
[ ] Create a summary in the .md file

Learning Objectives

By the end of this chapter, you should be able to:

        Discuss about SDN controllers.
        Explain the functions of SDN controllers.
        Review open source and commercial SDN controllers: OpenDaylight, ONOS, Cisco ACI, VMware NSX, Project Calico, Big Cloud Fabric, Tungsten Fabric.

Next, we will talk about SDN controllers. An SDN controller is a single hive or brain of the network that manages all the networking devices in its domain.

SDN Controller Cluster

Before diving into this chapter, it is good to clarify two things:

        Southbound Protocol
        This is the method used by an SDN controller to communicate with networking devices in its domain. Most significant protocols are OpenFlow, SNMP, CLI (Telnet/SSH), P4, as well as custom protocols.
        Northbound Protocol/API
        This is the method used by other software and tools (such as orchestration platforms) to communicate with the SDN controller. An SDN controller publishes its northbound APIs to allow other systems to communicate with the SDN controller.

It is very important that you understand the main technologies involved in software defined networking. These technologies are used by the SDN controllers to manage and control the SDN network. In general, there are two ways to control the fabric (underlying network):

        Direct Fabric Programming
        The SDN controller communicates directly with SDN-enabled switches via southbound protocols such as OpenFlow, NETCONF, OVSDB, or even the P4 runtime. The SDN controller programs each switch member with related information about Fabric, and advises how to forward the traffic. Direct Fabric programming is used by SDN controllers such as OpenDaylight, ONOS, and Floodlight.
        Overlay
        The SDN controller does not rely on programming the underlying network switches and routers. Instead, it builds a virtual overlay network on top of the existing underlay network. The underlying network can be an L2 or L3 network with traditional switches and routers, just providing IP connectivity. The SDN controller uses this platform to build the overlay using encapsulation protocols such as VXLAN and GRE. Tungsten, Project Calico and VMware NSX use this method to manage the data plane.

Below you can see a quick comparison between these two technologies:

Feature	Direct Fabric Programming	Overlay
Can work and co-exist with existing IP network	NO	YES
Required to encapsulate packets	NO	YES
Scalable	YES	YES

Knowledge Bridge

It is very important for a network engineer to understand what exactly the SDN controller is and how it is different from legacy networking concepts. To make it simple, imagine an SDN controller as a database that knows each and every endpoint and MAC address in a network, as well as the paths within the network that it manages. The SDN controller becomes a single brain that knows all endpoints, as well as knows how to route traffic to and from outside networks.

When it comes to legacy networking, each switch or router maintains its own database of MAC addresses and endpoints, builds its own topology view, and calculates the paths from its own perspective without having the full picture of the network. This usually leads to issues which most of us have been challenged with, such as traffic engineering, sub-optimal routing, loops and traffic back holes.

SDN controllers will help manage traffic routing the way businesses want, rather than the way dictated by a routing protocol while avoiding loops, sub-optimal routing, etc. SDN controllers manage the routing protocol communication with outside of the SDN domain, establish BGP neighbor relationship, and decide how traffic should get routed to the SDN domain, or routed outside.

Relationship between a Network Managed by an SDN Controller and External Networks

OpenDaylight (ODL) is:

"a modular open platform for customizing and automating networks of any size and scale".

It is a collaborative project hosted by The Linux Foundation. OpenDaylight is modular, scalable, and has numerous southbound protocols implemented. At its core is the Model-Driven Software Abstraction Layer (MD-SAL). OpenDaylight is based on Linux, uses Apache Karaf, and the OSGi platform to launch. The code is written in Java.

Networking functions in ODL are implemented as networking applications. They run on top of ODL. ODL allows these applications to manage the underlying network. Each application hooks to ODL to receive the networking event.

ODL - Quick Summary

Name | OpenDaylight (ODL) SDN controller By | The Linux Foundation Where it runs | As an SDN controller, it runs on a separate host or a cluster of nodes to manage the underlying network What it does | Manages and operates underlying network devices using southbound protocols Features | It comes with a comprehensive library of protocols, ready made applications. It is a modular and flexible system, and it allows you to develop any networking application. What it can do out-of-the-box | You can use ODL for networking with your cloud orchestration platform such as OpenStack. ODL can manage OpenFlow-based switches (it can be bare metal). You can develop a custom event-driven application on top of ODL. Normally, applications get activated on a specific event (for example, a packet arrived) and perform some actions.

ODL provides both a command line interface, as well as a web GUI.

What Applications Is ODL Shipped With?

ODL is shipped with many applications. Some of the core applications are:

        L2 Switch
        This is an implementation of Layer2 switching in a network managed by ODL. It uses OpenFlow as the southbound protocol. When a new host connects to a network switch (virtual or physical switch), the OpenFlow agent in the switch reports back to the SDN controller the MAC address newly learned from that port. ODL receives this information and adds it to its large table of information related to the network.
        BGP
        BGP is implemented as an application on top of ODL. BGP runs on the ODL controller, establishes peering with other neighbors and exchanges network prefixes. The main difference between a BGP running on the ODL controller and a BGP running on a standalone network router is that in a standalone network router or a switch, the BGP directly manipulates and modifies the routing table of the device. However, in an SDN controller, the BGP application stores and maintains a BGP table centrally, but it will inject the required entries to the forwarding table of multiple underlying switches.
        Traffic Engineering
        One useful feature of ODL is its ability for custom traffic engineering. Traffic engineering refers to how you would like the traffic to traverse your network, how you would like the entry and exit points of your network to behave. This is very important in service provider networks, where traffic needs to be routed on different paths to avoid congestion and saturation of the network links.
        Network Virtualization for Cloud Platforms
        Cloud platforms such as OpenStack rely on creating tenants (AKA Projects in OpenStack) for isolating the resources and network data traffic of multiple tenants from each other. ODL can integrate with OpenStack to create isolated networks for tenants (network slices). These network slices are not only a representation of a VLAN, but it's fully isolated within the whole SDN domain.

Building Applications for OpenDaylight

You can build your application based on standard traditional SAL (Service Adaptation Layer) or MD-SAL (Model-Driven SAL). Building applications with MD-SAL allows you to reuse your models later for other applications that you may want to build. However, SAL will require to use specific hard-coded libraries that will tie the application to a specific library. For example, you can build an application to use SNMP to gather MAC address tables from a switch. If later you need to collect the MAC address from a switch using a NETCONF, your application will not work. However, if you use MD-SAL, you rely on the OpenDaylight core to use the required protocol (NETCONF or SNMP) to query a switch. Each MD-SAL application requires some Java and YANG code to be deployed.

In general, an application or module starts with a YANG model. YANG is a language, or it may be better to say a format, to build the model. YANG is a markup language similar to JSON, but used mainly for modeling.

The YANG model file will be fed to YANG tools (such as the YANG interpreter engine of ODL) whenever it needs to generate other source files when we need to do JAVA coding.

The YANG tools engine automatically generates the application scripts and you can continue to code your application using the generated templates.

Creating Applications in ODL (Full Process)

Video

Open Network Operating System (ONOS)

The ONOS (Open Network Operating System) project is hosted by The Linux Foundation and maintained by ONF (Open Networking Foundation). Similar to OpenDaylight, ONOS is also written in Java and works on top of Apache Karaf OSGi. It can be deployed on a cluster of multiple servers to provide high availability.

ONOS supports multiple southbound protocols to program the underlying virtual or physical switches and routers. ONOS supports protocols such as OpenFlow, NETCONF, or OpenConfig, but the system is not tied to any of these protocols. Similar to OpenDaylight, ONOS also provides a high-level set of abstractions and models to the applications. Applications can use these models to communicate with an underlying network device (i.e. switch) without knowing what protocol is used for communication with that device. For example, when an application requires to get a list learned MAC addresses of a switch, it just calls an API in ONOS to get the list. ONOS may send an SNMP or NETCONF command to get the list of MAC addresses from the switch. However, the method of communication is completely transparent to applications and users.

There is no restriction to use the model abstraction method. Applications can directly use the southbound protocol libraries to communicate directly with a device. However, applications will be tied to use that protocol only.

Applications can be loaded and unloaded dynamically, without any need to restart the ONOS core servers. ONOS application management distributes the networking applications of all other nodes and servers in the cluster and ensures all servers in the cluster are running that application.

ONOS provides a web interface GUI, a CLI and REST APIs for management and integration with other systems.

ONOS - Quick Summary

Name	Open Network Operating System (ONOS)
By	Hosted by The Linux Foundation, maintained by the Open Networking Foundation
Where it runs	As an SDN controller platform, it runs on a separate host to manage the underlying network
What it does	Manages and operates the underlying network devices using southbound protocols
Features	Modular software that allows building plugins and applications, built-in northbound APIs such as REST and gRPC, built-in southbound protocols such as P4, OpenFlow, NETCONF, TL1, SNMP, CLI, BGP, RESTCONF. GUI, YANG tool chain
What it can do out-of-the-box	ONOS can be used as a standard SDN controller to manage underlying networking devices such as routers and switches. ONOS supports different southbound protocols, such as OpenFlow, BGP, and NETCONF, which can configure and manage underlying routers and switches.

ONOS Commands

In addition to the web GUI, you can use the onos command to manage and operate ONOS:

onos> help onos COMMANDS onos:add-flows Installs a flow rule onos:add-host-intent Installs host-to-host connectivity intent onos:add-multi-to-single-intent Installs point-to-point connectivity intent onos:add-node Adds a new controller cluster node onos:add-optical-intent Installs optical connectivity intent

ONOS vs ODL

ONOS and ODL both use similar concepts and infrastructure to run the SDN controller. For example, both use Apache Karaf OSGi with a modular approach towards applications; they also use southbound protocols.

The main difference between ONOS and ODL is that ONOS is more Telcos and Service provider-oriented, and ODL is mode cloud and datacenter-oriented.

	ODL	ONOS
Applications for datacenters, cloud environment	ODL provides applications to integrate with OpenStack (such as VTN Virtual Tenant Manager) or southbound protocols that are used in datacenters, such as OVSDB	SONA (Simplified Overlay Network Architecture) is an ONOS application which integrates with OpenStack and provides network virtualization and tenant isolation
Applications for service providers, telcos	Limited to some use cases and southbound protocols, such as BGP-PCEP	Has many use cases and applications for telcos, such as CORD, Packet Optical, IP RAN, SDN IP, VPLS, Carrier Ethernet, etc.
Performance and high availability	ODL supports clustering	ONOS clustering is mature and comprehensive

Tungsten Fabric (formerly OpenContrail)

Tungsten Fabric is a new collaborative project hosted by The Linux Foundation. Tungsten Fabric was born when Juniper's OpenContrail project moved to The Linux Foundation in March 2018. The broader mission of the Tungsten Fabric project is to make Tungsten Fabric a networking fabric for virtual and containerized workloads. Tungsten Fabric is a network virtualization platform and an overlay controller. Its main role is to provision isolated virtual networks to be mainly used within a cloud datacenter.

Tungsten Fabric - Quick Summary

Name	Tungsten Fabric (formerly OpenContrail)
By	The Linux Foundation
Where it runs	On multiple nodes; it can be deployed on each compute host of a virtual environment
What it does	Creates and manages virtual overlay networks over any underlying fabric
Features	Layer2 and Layer3 overlays over a Layer3 underlay. Uses MPLS over GRE/MPLS over UDP. The underlying switches do not need to support MPLS
What it can do out-of-the-box	Tungsten Fabric works well in a cloud environment, such as OpenStack. You can deploy and integrate Tungsten Fabric with your cloud orchestration software to provide services such as service function chaining and tenant network isolation.

One of the major differences between Tungsten Fabric and other network controllers that we have looked at in this chapter is that Tungsten Fabric’s forwarding plane is only limited to hosts. This means that Tungsten Fabric can only manage networking on hosts, not on physical routers and switches.

Virtual Overlay Networks Over A Standard Legacy L2/L3 Network

Tungsten Fabric consists of the following two main components:

        Controller
        The controller is a bundle of multiple applications and roles (such as a database, controller, configurator) and a web user interface to manage Tungsten Fabric. The controller can be fully clustered and highly available in different locations.
        vRouter
        It builds up the Tungsten Fabric data plane. vRouter is a kernel module that can be loaded on the Linux kernel of the hypervisor hosts.  vRouter is similar to the Linux bridge in the kernel. vRouter has additional duties, such as encapsulation and decapsulation of tunneled traffic. Also, vRouter is MPLS-aware, can add and remove MPLS tags on packets, and, more importantly, can route the MPLS packets based on their tags to the relevant routing instance.

To allow communication with devices outside of the virtual network, Tungsten Fabric requires a gateway device to communicate with the external network. A router with BGP L3VPN capabilities or a virtual router can be used as a software gateway.

Tungsten Fabric has multiple use cases in network virtualization and service chaining. Private clouds, service providers, Infrastructure as a Service (IaaS) and Virtual Private Clouds (VPCs) can leverage benefits from Tungsten Fabric to build a virtual network for their cloud environments. Service providers that offer Private Cloud, Virtual Private Cloud (VPC), or Infrastructure as a Service (IaaS) require a networking platform that can support multi-tenant network virtualization. In such environments, there are different tenants being hosted and using the compute, network and storage services. In many cases, the virtual network of tenants overlaps with each other (IP overlapping). Tungsten Fabric can provide a full network virtualization and isolation between tenants while tenant resources are located on different compute hosts in different locations.

Tungsten Fabric: Virtual Networks

Virtual overlay networks are the key concept in Tungsten Fabric. A virtual network is similar to a VLAN in Layer 2 and a VRF in Layer 3. It is a fully isolated network with no Layer 2 or Layer 3 reachability to other virtual networks, unless it is defined and permitted.

In a cloud environment, each tenant can have one or more virtual networks. Virtual networks are isolated from each other by a security policy.

Tungsten Fabric uses MPLS for network isolation. However, it doesn’t send any MPLS-labeled packet on the underlay network. All MPLS packets are encapsulated in GRE or UDP and are sent between the compute hosts. Compute hosts which are running the Tungsten Fabric vRouter will decapsulate the MPLS packet out of the UDP or GRE packet. Each virtual network in Tungsten Fabric has its own MPLS label. The vRouter sends the packet to the correct vRouter instance based on the MPLS tag.

Tungsten Fabric Architecture

Tungsten Fabric vRouter Architecture within a Compute Host

Tungsten Fabric: Service Chaining

Service chaining is a network-related process to forward traffic to specific network services such as a firewall or an IPS prior to forwarding it to the final destination. Service chaining can be implemented using policy routing in legacy networking or via SDN policies within a modern SDN network.

Tungsten Fabric provides a policy language which can be used to define how the virtual networks can communicate with each other. If you are familiar with Snort IDS, you will find the language similar to Snort policies.

The policy looks like:

Allow any vn1 -> vn2 svc-1, svc-2

The above policy means that traffic from virtual network 1 (vn1) to virtual network 2 (vn2) is allowed and should pass first to service 1 (svc-1) and to service 2 (svc-2), and then to virtual network 2.

Tungsten Fabric created additional routing instances for servicing virtual machines automatically, and will use them for service chaining and sending traffic via the service hosts.

Service hosts are mainly IPS, IDS. Tungsten Fabric performs service chaining by:

        Manipulating the route targets to influence importing and exporting routing between routing instances
        Manipulating the next-hops or labels of the routes as they are leaked between routing instances to force traffic through the sequence of services that are defined in the policy.

Service Chaining Policy Applied for Traffic between the Two Virtual Machine Workloads

Project Calico

Project Calico is an open source initiative for network virtualization and SDN. It provides secure network connectivity for containers and virtual machines.

Project Calico avoids using overlays (encapsulations), as they believe they reduce performance. Instead, Project Calico has an IPAM (IP Address Management) module which tries to allocate IP addresses for hosts and workloads (containers, virtual machines) to ensure they are all routable. This method reduces the chance of any encapsulation requirement. However, if really required, Project Calico performs IP-in-IP tunneling.

Project Calico - Quick Summary

Name	Project Calico
By	Calico open source community, backed by Tigera, Inc.
Where it runs	The Calico controller runs on multiple nodes, and gets deployed on each compute host of a virtual environment
What it does	Creates and manages virtual networks for VMs, containers and bare metal
Features	Supports overlay and encapsulation, uses a built-in IPAM for allocation and assigning IP addresses to workloads
What it can do out-of-the-box	Project Calico can be used in OpenStack, Kubernetes, and other orchestration environments to provide networking

Comparing Project Calico to Tungsten Fabric, they both are very similar. The main difference is that Tungsten Fabric uses MPLS and overlays, while Project Calico avoids using overlays, and instead uses routable IP addresses to reach to each workload. In most cases, Project Calico routes the traffic from the underlying network to the workloads (virtual machines or containers) without tunneling or encapsulation. In case an overlay is required, Project Calico uses VXLAN or IP-in-IP tunneling methods.

To secure the networks, Project Calico provides dynamic enforcement of network security rules. Using a simple policy language, you can define policies to control communication between containers, virtual machine workloads, and bare metal host endpoints.

The following diagram illustrates the architecture of Project Calico:

he Calico DataPlane consists of compute hosts. Felix is an agent software that runs on each host and programs the kernel routes to local workloads, as well as enforces the filtering rules that are required by defined policies applied to each tenant or specific workloads. The routing components manage communication with other nodes to ensure IP reachability between all nodes and workloads within the domain.

Calico integrates with orchestrator tools such as OpenStack and Kubernetes using a plugin software. This plugin interfaces with the orchestrator and listens for events related to the creation and deletion of workloads (virtual machines or containers). Once Calico receives an event related to the creation of a new workload, it can allocate and assign IP addresses to the new workload, as well as record the MAC address of the new workload and where it is located.

The workload’s identity, IP addresses, and their network policy are shared between all nodes within the domain. Calico can use etcd for this function, or it can use the Kubernetes API data store when working in a Kubernetes environment.

Cisco Application-Centric Infrastructure (ACI)

The Cisco ACI (also known as the Cisco Application Policy Infrastructure) is a commercial SDN controller which works with the Cisco datacenter ecosystem, including Cisco Nexus 9000 switches, Cisco UCS (Unified Computing System), and their converged infrastructure platform. ACI has some capabilities to communicate with virtual switches such as VMware vSwitch or Cisco 100v virtual switch.

In order to have ACI manage a Cisco Nexus 9000 switch, the switch must run the NX-OS in the ACI mode, that is a lightweight agent software on Cisco Nexus which allows the ACI to communicate, program and manage the forwarding tables of the Cisco Nexus switch.

ACI uses a southbound protocol called OpFlex to program and manage the Nexus switches. OpFlex is an open source protocol similar to OpenFlow, but with some differences.

ACI - Quick Summary Name	Cisco ACI SDN Controller
By	Cisco Systems
Where it runs	ACI/APIC runs on a cluster of servers to manage the underlying network of Nexus 9000 switches
What it does	Manages the SDN domain and its switches. Provides traffic isolation, security, and virtual networks.
Features	Manages single and multiple sites
What it can do out-of-the-box	You can integrate the ACI with your virtualization platform or cloud environment and manage your physical and virtual network switches. ACI provides automation features that can help enhance the processes and reduce the time for changes in the network.

Cisco ACI is mainly used in datacenter and cloud infrastructures. Cisco also has another SDN platform for service providers called NSO (Network Services Orchestration), which is a product designed for SDN for service providers which can manage not only the Cisco underlay equipment, but also the equipment from other vendors.

Cisco APIC is a modular platform with pluggable southbound and northbound plugins that can manage different platforms.

Cisco ACI has integration capabilities with third-party systems such as:

        Load balancers
        VMware vSphere
        Kubernetes
        Microsoft Hyper-V, System Center Virtual Machine Manager (SCVMM)
        OpenStack.

Floodlight

Floodlight is another open source SDN controller, sponsored and maintained by Big Switch Networks. Floodlight is not just an SDN controller, but can be used with OpenFlow to write applications on top of the Floodlight controller.

Floodlight - Quick Summary Name	Floodlight Controller
By	Sponsored by Big Switch Networks
Where it runs	On a cluster of hosts to manage an underlay network
What it does	Floodlight is a modular SDN controller capable of having SDN applications. It manages the underlying physical and virtual switches via the OpenFlow protocol.
Features	Integrates with OpenStack; open source
What it can do out-of-the-box	Floodlight documentation is not fancy and might seem old. However, you can use Floodlight to manage any OpenFlow-capable physical switch.

Floodlight includes applications such as Firewall, Load balancer, L2 Switch, etc.

Floodlight supports some OpenFlow-enabled switches, such as Arista, Open Virtual Switch (OVS), Brocade, Dell, HP, Huawei, Pica8, Juniper, etc.

Big Cloud Fabric (BCF)

Big Switch Networks is one of the commercial SDN solution providers in the market. Big Cloud Fabric (BCF) is their SDN product.

Big Cloud Fabric - Quick Summary

Name	Big Cloud Fabric (BCF) By	Big Switch Networks
Where it runs	On a cluster of servers. To manage the data plane, Big Cloud Fabric provides a lightweight NOS called SwitchLight (to be installed on a compatible hardware)
What it does	Manages a datacenter or tenant-based cloud environment
Features	Commercial SDN controller, works with bare metal switches, integrates with cloud orchestration tools such as OpenStack, VMware vSphere, and Kubernetes.
What it can do out-of-the-box	By deploying Big Cloud Fabric, you can integrate a network with your cloud orchestration platform. For example, BCF integrates with VMware vSphere. Any VLAN you create on VMware vSphere will be automatically presented in your underlying network managed by BCF. BCF can apply network policies and service chaining on your network traffic to enforce traffic to pass through a network analytics or IPS when it is being routed to the Internet.

Big Cloud Fabric has a REST API engine which can be used for management; it also has an industry standard CLI, which technically calls the APIs of the Big Cloud Fabric platform.

Big Cloud Fabric uses white box bare metal switches as its hardware forwarding plane. BCF includes a lightweight NOS for bare metal switches, which runs an agent software on switches to get managed by the BCF controller.

BCF integrates with cloud orchestration platforms such as OpenStack, Kubernetes, Nutanix, VMware vSphere, etc. Once integrated, BCF listens for events such as new workload creation, migration, destruction, etc., and updates the forwarding plane accordingly. BCF is tenant-aware, and ensures the network traffic from different tenants are not mixed together.

To manage the virtual switches in compute nodes, BCF has a plugin that gets loaded on compute nodes. That plugin establishes communication with the BCF controller to manage routes and policies within a virtual switch in a compute host. This allows BCF to fully manage the virtual and the physical underlying networks.

BCF uses direct fabric programming (control plane) to manage the packet routing and switching within the SDN domain. BCF supports service chaining and insertion across for the whole SDN domain, or for a specific tenant.

Big Switch Networks also offers a community version of Big Cloud Fabric, which is limited to only 1 switch and managing up to 600 endpoints, 200 VLANs and 50 Tenants.

Learning Objectives (Review)

You should now be able to:

        Discuss about SDN controllers.
        Explain the functions of SDN controllers.
        Review open source and commercial SDN controllers: OpenDaylight, ONOS, Cisco ACI, VMware NSX, Project Calico, Big Cloud Fabric, Tungsten Fabric.

Summary

Project/Product	License Type	SDN Method	Main Use Case
OpenDaylight	Open Source	Direct Fabric Programming	Cloud, datacenter, multi-tenant
ONOS	Open Source	Direct Fabric Programming	Telco, service providers
Tungsten Fabric	Open Source	Overlay	Cloud, datacenter, multi-tenant
Project Calico	Open Source	Overlay	Cloud, datacenter, multi-tenant
Cisco ACI	Commercial	Direct Fabric Programming	Cross sites, cloud, datacenter, multi-tenant
Floodlight	Open Source	Direct Fabric Programming	Cloud, datacenter, multi-tenant
Big Cloud Fabric	Commercial	Direct Fabric Programming	Cloud, datacenter, multi-tenant

readersclub / opensource-networking-technologies

Chapter 6. Network Control #7

ODL - Quick Summary

Big Cloud Fabric - Quick Summary