use ClusterAutoscaler + tolerations + nodeAffinity + podAntiAffinity

[AkihiroSuda]

If we can use Cluster Autoscaler, we don't need to have home-made IaaS driver in this repo.

We could probably also use tolerations + nodeAffinity + podAntiAffinity along with Cluster Autoscaler. (but have performance overhead?)

[AkihiroSuda]

apiVersion: apps/v1
kind: Deployment
metadata:
  name: foo
  labels:
    app: foo
    ipp-class: "class0"
spec:
  selector:
    matchLabels:
      app: foo
  template:
    metadata:
      labels:
        app: foo
        ipp-class: "class0"
    spec:
      containers:
      - name: nginx
        image: nginx:alpine
      tolerations:
      - key: ipp
        operator: Equal
        value: "true"
        effect: NoSchedule
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: ipp
                operator: In
                values: ["true"]
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: ipp-class
                operator: NotIn
                values: ["class0"]
            topologyKey: kubernetes.io/hostname

# GKE node pool configuration:
# * Enable autoscaling (minimum number of nodes must be >= 1)
# * Add node label: "ipp" = "true"
#   (GKE nodepool doesn't support node labels like "ipp.node-restriction.kubernetes.io/ipp" ?)
# * Add node taint: "ipp" = "true" , NoSchedule
# * Create pods like this YAML.
#   A node is automatically added when a pod with new "ipp-class" label value is created.