fmuyassarov
commented
1 year ago It feels like you are missing Role and RoleBinding.
Usually, when I deploy balloons plugin via kustomize I get Role and Rolebinding as well
$ kustomize build deployment/overlays/balloons/ | kubectl apply -f -
customresourcedefinition.apiextensions.k8s.io/noderesourcetopologies.topology.node.k8s.io created
serviceaccount/nri-resource-policy created
role.rbac.authorization.k8s.io/nri-resource-policy created
clusterrole.rbac.authorization.k8s.io/nri-resource-policy created
rolebinding.rbac.authorization.k8s.io/nri-resource-policy created
clusterrolebinding.rbac.authorization.k8s.io/nri-resource-policy created
configmap/nri-resource-policy-config.default created
daemonset.apps/nri-resource-policy createdAnd in the generated file you can see the Role and Rolebinding. So, could it be that in the TA plugin, you have the role and rolebinding?
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
api-approved.kubernetes.io: https://github.com/kubernetes/enhancements/pull/1870
controller-gen.kubebuilder.io/version: v0.11.2
creationTimestamp: null
name: noderesourcetopologies.topology.node.k8s.io
spec:
group: topology.node.k8s.io
names:
kind: NodeResourceTopology
listKind: NodeResourceTopologyList
plural: noderesourcetopologies
shortNames:
- node-res-topo
singular: noderesourcetopology
scope: Cluster
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
description: NodeResourceTopology describes node resources and their topology.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
topologyPolicies:
items:
type: string
type: array
zones:
description: ZoneList contains an array of Zone objects.
items:
description: Zone represents a resource topology zone, e.g. socket,
node, die or core.
properties:
attributes:
description: AttributeList contains an array of AttributeInfo objects.
items:
description: AttributeInfo contains one attribute of a Zone.
properties:
name:
type: string
value:
type: string
required:
- name
- value
type: object
type: array
costs:
description: CostList contains an array of CostInfo objects.
items:
description: CostInfo describes the cost (or distance) between
two Zones.
properties:
name:
type: string
value:
format: int64
type: integer
required:
- name
- value
type: object
type: array
name:
type: string
parent:
type: string
resources:
description: ResourceInfoList contains an array of ResourceInfo
objects.
items:
description: ResourceInfo contains information about one resource
type.
properties:
allocatable:
anyOf:
- type: integer
- type: string
description: Allocatable quantity of the resource, corresponding
to allocatable in node status, i.e. total amount of this
resource available to be used by pods.
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
available:
anyOf:
- type: integer
- type: string
description: Available is the amount of this resource currently
available for new (to be scheduled) pods, i.e. Allocatable
minus the resources reserved by currently running pods.
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
capacity:
anyOf:
- type: integer
- type: string
description: Capacity of the resource, corresponding to capacity
in node status, i.e. total amount of this resource that
the node has.
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
name:
description: Name of the resource.
type: string
required:
- allocatable
- available
- capacity
- name
type: object
type: array
type:
type: string
required:
- name
- type
type: object
type: array
required:
- topologyPolicies
- zones
type: object
served: true
storage: false
- name: v1alpha2
schema:
openAPIV3Schema:
description: NodeResourceTopology describes node resources and their topology.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
attributes:
description: AttributeList contains an array of AttributeInfo objects.
items:
description: AttributeInfo contains one attribute of a Zone.
properties:
name:
type: string
value:
type: string
required:
- name
- value
type: object
type: array
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
topologyPolicies:
description: 'DEPRECATED (to be removed in v1beta1): use top level attributes
if needed'
items:
type: string
type: array
zones:
description: ZoneList contains an array of Zone objects.
items:
description: Zone represents a resource topology zone, e.g. socket,
node, die or core.
properties:
attributes:
description: AttributeList contains an array of AttributeInfo objects.
items:
description: AttributeInfo contains one attribute of a Zone.
properties:
name:
type: string
value:
type: string
required:
- name
- value
type: object
type: array
costs:
description: CostList contains an array of CostInfo objects.
items:
description: CostInfo describes the cost (or distance) between
two Zones.
properties:
name:
type: string
value:
format: int64
type: integer
required:
- name
- value
type: object
type: array
name:
type: string
parent:
type: string
resources:
description: ResourceInfoList contains an array of ResourceInfo
objects.
items:
description: ResourceInfo contains information about one resource
type.
properties:
allocatable:
anyOf:
- type: integer
- type: string
description: Allocatable quantity of the resource, corresponding
to allocatable in node status, i.e. total amount of this
resource available to be used by pods.
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
available:
anyOf:
- type: integer
- type: string
description: Available is the amount of this resource currently
available for new (to be scheduled) pods, i.e. Allocatable
minus the resources reserved by currently running pods.
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
capacity:
anyOf:
- type: integer
- type: string
description: Capacity of the resource, corresponding to capacity
in node status, i.e. total amount of this resource that
the node has.
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
name:
description: Name of the resource.
type: string
required:
- allocatable
- available
- capacity
- name
type: object
type: array
type:
type: string
required:
- name
- type
type: object
type: array
required:
- zones
type: object
served: true
storage: true
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: nri-resource-policy
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: nri-resource-policy
namespace: kube-system
rules:
- apiGroups:
- ""
resources:
- configmaps
verbs:
- get
- watch
- apiGroups:
- topology.node.k8s.io
resources:
- noderesourcetopologies
verbs:
- create
- get
- list
- update
- delete
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: nri-resource-policy
rules:
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: nri-resource-policy
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: nri-resource-policy
subjects:
- kind: ServiceAccount
name: nri-resource-policy
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: nri-resource-policy
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: nri-resource-policy
subjects:
- kind: ServiceAccount
name: nri-resource-policy
namespace: kube-system
---
apiVersion: v1
data:
policy: |
ReservedResources:
cpu: 750m
#balloons:
kind: ConfigMap
metadata:
name: nri-resource-policy-config.default
namespace: kube-system
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
labels:
app: nri-resource-policy
name: nri-resource-policy
namespace: kube-system
spec:
selector:
matchLabels:
app: nri-resource-policy
template:
metadata:
labels:
app: nri-resource-policy
spec:
containers:
- args:
- --host-root
- /host
- --fallback-config
- /etc/nri-resource-policy/nri-resource-policy.cfg
- --pid-file
- /tmp/nri-resource-policy.pid
- -metrics-interval
- 5s
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
image: nri-resource-policy-balloons:devel
imagePullPolicy: Always
name: nri-resource-policy-balloons
ports:
- containerPort: 8891
hostPort: 8891
name: metrics
protocol: TCP
resources:
requests:
cpu: 500m
memory: 512Mi
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
volumeMounts:
- mountPath: /var/lib/nri-resource-policy
name: resource-policydata
- mountPath: /host/sys
name: hostsysfs
- mountPath: /var/run/nri-resource-policy
name: resource-policysockets
- mountPath: /etc/nri-resource-policy
name: resource-policyconfig
- mountPath: /var/run/nri
name: nrisockets
nodeSelector:
kubernetes.io/os: linux
serviceAccount: nri-resource-policy
volumes:
- hostPath:
path: /var/lib/nri-resource-policy
type: DirectoryOrCreate
name: resource-policydata
- hostPath:
path: /sys
type: Directory
name: hostsysfs
- hostPath:
path: /var/run/nri-resource-policy
name: resource-policysockets
- configMap:
name: nri-resource-policy-config
name: resource-policyconfig
- hostPath:
path: /var/run/nri
type: Directory
name: nrisockets
changzhi1990
Hi, all
I meet an error during use the balloon policy in NRI. Here is the detailed steps:
After that, I check the logs of the NRI pod:
The error message is failed to look up current node resource topology CR: noderesourcetopologies.topology.node.k8s.io "xx-icx-2" is forbidden: User "system:serviceaccount:kube-system:nri-resource-policy" cannot get resource "noderesourcetopologies" in API group "topology.node.k8s.io" at the cluster scope.
But this error doesn't exist in the topology-aware policy.
So does the error impact the balloon policy or we can just ignore it?