Add Documentation To Calico CNI Type Options

[aauren]

Expected Behavior

Documentation should fully describe what the supported Calico CNI types do and why you would use them.

Current Behavior

Documentation currently doesn't do a good job explaining what the host-local type does or why you would or wouldn't want to use it over the calico-ipam type. From the documentation it only says the following:

When using the CNI host-local IPAM plugin, a special value usePodCidr is allowed for the subnet field. This tells the plugin to determine the subnet to use from the Kubernetes API based on the Node.podCIDR field.

node_name The node name to use when looking up the usePodCidr value (defaults to current hostname)

When making use of the usePodCidr option, the Calico CNI plugin requires read-only Kubernetes API access to the Nodes resource.

Possible Solution

Flesh out the documentation section a bit more to include information on what the options are, and specifically what changes are introduced to the way Calico works when the different options are exercised. It would also be helpful to give examples as to why you would use one or the other, and perhaps even specifically state that one should be the default option in most use cases.

@caseydavenport let me know more about the options in Slack:

Yeah, there are a few differences between Calico IPAM and host-local. One of the main ones is where they store their state - Calico IPAM uses etcd and host-local uses files on disk.

The reason you'd want to use host-local instead of Calico IPAM is mostly if you're using Calico in its "Kubernetes API datastore" mode instead of having Calico store state in etcd.

I think that provides a place to start, but any additional information would be helpful.

Steps to Reproduce (for bugs)

N/A

Context

I spent several hours combing through documentation before I had to ask @caseydavenport in slack.

Your Environment

• Calico version: 3.0
• Orchestrator version (e.g. kubernetes, mesos, rkt): kubernetes

CC @emanic (as requested by @caseydavenport)

[caseydavenport]

@emanic I think this is a good example of an area where we're missing a lot of key information (CNI docs in general). Would be good to brainstorm how / where we can fit this sort of information in.

[aauren]

This doesn't have to do specifically with CNI, but while I was trying to trace down some etcdv3 compaction errors that I was experiencing with my calico-kube-controller deployment. I found some more configuration options that are undocumented here: https://github.com/projectcalico/kube-controllers/blob/master/pkg/config/config.go

Specifically, I needed to know that I could stop Calico from doing it's own compaction by setting the env variable COMPACTION_PERIOD to 0.