Support custom annotations on the Ceph dashboard service

[sfackler]

Is this a bug report or feature request?

• Feature Request

What should the feature do: CephCluster should allow users to add custom annotations to the Service object that Rook generates for the Ceph dashboard. Following the example of CephObjectStore's gateway configuration, I'm imagining:

spec:
  dashboard:
    annotations:
      my.annotation: foobar

What is use case behind this feature: Similarly to the motivations for annotations on the object store gateway service, this is useful when running Ceph in a cluster with controllers that expect specific service annotations.

I'm happy to make the PR if this seems reasonable.

[travisn]

There are currently mgr annotations that can be applied to the mgr pod, but not on the dashboard svc as noted. The annotations on the mgr pod are applied as such in the CephCluster CR, and we could add a dashboard section:

spec:
  annotations:
    mgr:
      - key: val
    dashboard:
      - key: val

Likewise, perhaps dashboard labels could be added to the service.

This should be a simple addition to the makeDashboardService() method, sounds great if you could open a PR. No changes to the CRD schema should be necessary since the annotations are defined as a map. See also annotations.go, and callers of that code for example usage.