Making testing Gatekeeper easy

Given that the architecture of Gatekeeper favors scalability, it requires several machines to build a testbed. This demand makes experimenting with Gatekeeper difficult. Using virtual machines (VMs) lowers this demand, but VMs do not eliminate it. One still has to run Gatekeeper and Grantor servers on real machines because Gatekeeper does not work with virtual NICs; virtual NICs do not support essential features like multi-queues and RSS that enable Gatekeeper to go fast.

A solution to further lower the effort required to experiment with Gatekeeper is to employ Linux's network namespaces. One needs a big machine with double the number of NICs that Gatekeeper and Grantor servers require, connects these pairs of NICs with a loopback cable, and establishes the needed topology employing network namespaces. One sets a network namespace for each machine being virtualized, connects these namespaces using pairs of virtual NICs, and uses the pairs of real NICs to connect the namespaces of the Gatekeeper and Grantor servers to the other namespaces.

The setup proposed above is still easier to maintain than employing VMs because a single machine is maintained, namely, the big machine that runs all namespaces. Thus, for example, if the kernel is updated or a new tool is installed, the newer kernel and the new tool are immediately available to all namespaces.

With this new testbed approach, we can port the Amazon AWS experiment developed to test Gatekeeper (see our technical report) to use hardware NICs, avoid encapsulating packets, enable VLAN, and enforce point-to-point interconnections between the nodes. Once automated, we could run this complete Gatekeeper experiment to squash bugs before releases.

Once Gatekeeper supports virtual NICs, we can implement the complete Gatekeeper experiment described above as a GitHub action to run on every pull request and merge. This GitHub action would bring the code quality of Gatekeeper to a much higher standard. Not to mention that, at this point, people would have an easy time experimenting with Gatekeeper on their laptops.

Virtual NICs would also enable us to run experiments with multiple Gatekeeper and Grantor servers, having the request channel go through skbprio, and load balancing Grantor servers. We could parametrize this enlarged experiment to enable or disable features like all types of interface bonding, VLAN, IPv4, IPv6, Grantors in the same subnet of destinations or connected to routers, etc. We hunt for obscure bugs by scheduling our test server to run random parameters of this comprehensive test whenever nobody is using the server.

Finally, we can lower the implementation effort to bring this vision into reality by embracing Mininet to set things up. Mininet has only been used to teach computer networks, so it will likely require several patches to support all we need here, but writing these patches should be less demanding than implementing everything from scratch.

AltraMayor / gatekeeper

Making testing Gatekeeper easy #528