Include `app_guid` and `process_type` on app StatefulSet/Pods

tcdowney commented 5 years ago

tl;dr

We would like for app_guid and process_type to be included on the StatefulSet/Pod templates for Eirini apps to have deterministic and stable labels to select off of. E.g.

labels:
  guid: <capi-process-guid>
  process_type: <capi-process-type>           <--------------- NEW!
  app_guid: <capi-app-guid>                   <--------------- NEW!
  rootfs-version: ""
  source_type: APP
  version: <capi-process-version>

Background

The CF Networking Program is working on enhancing the routing tier for CF on Kubernetes (see: https://github.com/cloudfoundry-incubator/eirini/issues/72). We're currently working on fetching v3 Routes and v3 Destinations from CAPI and converting these into "CF Route" CRDs. These will be used to create Istio resources and Services to power an Istio/Envoy based routing tier for Eirini apps. See this design document for more information.

Problem

Route Destinations in Cloud Controller are currently a combination of route_guid, app_guid, and process_type. CAPI Process GUIDs are not stable over time (for example a rolling app deployment changes them) so this mapping allows for the Destination object to remain stable.

Since we currently only have a Process GUID on the StatefulSet/Pod for an app we'd have to do additional lookups on CAPI the Process GUID for a given app/process type and update our Services/Istio resources whenever an app's process GUID changes. Both of these are doable, but puts unnecessary load on the system when we could just have long lived resources that match off of the stable app guid and process type for the workload. This has the added benefit of giving other teams more stable identifiers to key off of.

Proposal

We enhance CAPI to send the app_guid and process_type along when it desires an app on Eirini and update Eirini to include this information as labels on the StatefulSet/Pod template.

Thanks! cc @ndhanushkodi @rosenhouse

Misc

Example StatefulSet (current)

Including this truncated StatefulSet to give folks an idea of what is on it currently.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  annotations:
    application_id: 150866b1-bf6a-45b7-abe8-d861cc26ae17
    application_name: zdt-test
    ...
    process_guid: 3dc6e687-0e73-44b0-8785-01cba02034f1-403850e1-3e17-4ae3-9868-ed832697b201
    ...
    space_name: s
    version: 403850e1-3e17-4ae3-9868-ed832697b201
  creationTimestamp: 2019-09-27T16:54:01Z
  generation: 1
  labels:
    guid: 3dc6e687-0e73-44b0-8785-01cba02034f1
    rootfs-version: ""
    source_type: APP
    version: 403850e1-3e17-4ae3-9868-ed832697b201
  name: zdt-test-s-b480caa13e
  ...
  resourceVersion: "8724193"
  ...
  uid: 680aee46-e147-11e9-bbfa-42010a000b0f
spec:
  podManagementPolicy: Parallel
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      guid: 3dc6e687-0e73-44b0-8785-01cba02034f1
      source_type: APP
      version: 403850e1-3e17-4ae3-9868-ed832697b201
  serviceName: ""
  template:
    metadata:
      annotations:
        application_id: 150866b1-bf6a-45b7-abe8-d861cc26ae17
        process_guid: 3dc6e687-0e73-44b0-8785-01cba02034f1-403850e1-3e17-4ae3-9868-ed832697b201
      creationTimestamp: null
      labels:
        guid: 3dc6e687-0e73-44b0-8785-01cba02034f1
        rootfs-version: ""
        source_type: APP
        version: 403850e1-3e17-4ae3-9868-ed832697b201
    spec:
      automountServiceAccountToken: false
      ...

cf-gitbot commented 5 years ago

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/168812438

The labels on this github issue will be updated when the story is started.

zrob commented 5 years ago

mix of underscores, dashes, and camel casing. oh my!

tcdowney commented 5 years ago

To @zrob's point, I went with the underscores because that seemed most consistent with the existing metadata on these resources. Happy to adapt to whatever convention y'all are using -- I imagine it's mostly underscored because that's how many of these fields are in VCAP_APPLICATION and in CAPI?

rosenhouse commented 5 years ago

This is done, right?

alex-slynko commented 5 years ago

Hi Yes, this was done as part of 1.0 release. We have added cloudfoundry.org/process_type and cloudfoundry.org/app_guid

cloudfoundry / eirini