Closed adaumann closed 1 year ago
If using the injector same issue:
Thanks we will take a look into this issue!
Hi @adaumann - Based on the error message in the last picture it appears as if your helm chart is out of date and behind the consul-k8s
binary, can you confirm that you're also using the helm chart 0.48.0
?
Hi @adaumann - Based on the error message in the last picture it appears as if your helm chart is out of date and behind the
consul-k8s
binary, can you confirm that you're also using the helm chart0.48.0
?
Yes I'm using 0.48.0 but with the redhat UBI image. The only version I hadd success is 0.43, but only to start an unsecure consul. (No Acl, no TSL, no gossip encryption). See other issue.
@adaumann How are you deploying OpenShift 4.11 right now? Do you have a link to docs on how to get OpenShift 4.11 up and running quickly?
Hello, I'm running OpenShift 4.11 as local single-node Kubernetes Cluster in Windows HyperV. In order to run it locally you can use the CRC version for developing. Just register in RedHat and look for "local" and download the installer. Currently there is no storage class defined by default, but you can register a "hostpath" local folder.
install.ps1:
# Install CRC from https://console.redhat.com/openshift/create/local
# Create pull secret from Redhat
# HyperV must be enabled
# Stop IIS, e.g. issreset /stop
crc setup
crc oc-env
crc start -m 20000 -c 8 -p .\pull-secret.txt
Sleep 10
$cr=crc console --credentials -o json | ConvertFrom-Json
$passwd=$cr.clusterConfig.adminCredentials.password
# CRC has normally no storage class for dynamic provision. At hostpath for testing
oc login -u kubeadmin -p $passwd https://api.crc.testing:6443
oc new-project local-path-storage
oc create serviceaccount local-path-provisioner-service-account -n local-path-storage
oc adm policy add-scc-to-user hostaccess -z local-path-provisioner-service-account -n local-path-storage
oc apply -f ./crc-hostpath.yaml
crc status
crc-hostpath.yaml:
#$ Adds storage class as Hostpath for CRC installations
#$ oc login -u kubeadmin -p <passwd> https://api.crc.testing:6443
#$ oc new-project local-path-storage
#$ oc create serviceaccount local-path-provisioner-service-account -n local-path-storage
#$ oc adm policy add-scc-to-user hostaccess -z local-path-provisioner-service-account -n local-path-storage
#$ oc apply -f crc-hostpath.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: local-path-provisioner-role
rules:
- apiGroups: [""]
resources: ["nodes", "persistentvolumeclaims"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["endpoints", "persistentvolumes", "pods"]
verbs: ["*"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "patch"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: local-path-provisioner-bind
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: local-path-provisioner-role
subjects:
- kind: ServiceAccount
name: local-path-provisioner-service-account
namespace: local-path-storage
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: local-path-provisioner
namespace: local-path-storage
spec:
replicas: 1
selector:
matchLabels:
app: local-path-provisioner
template:
metadata:
labels:
app: local-path-provisioner
spec:
serviceAccountName: local-path-provisioner-service-account
containers:
- name: local-path-provisioner
image: rancher/local-path-provisioner:v0.0.12
imagePullPolicy: IfNotPresent
command:
- local-path-provisioner
- --debug
- start
- --config
- /etc/config/config.json
volumeMounts:
- name: config-volume
mountPath: /etc/config/
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
volumes:
- name: config-volume
configMap:
name: local-path-config
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: local-path
provisioner: rancher.io/local-path
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete
---
kind: ConfigMap
apiVersion: v1
metadata:
name: local-path-config
namespace: local-path-storage
data:
config.json: |-
{
"nodePathMap":[
{
"node":"DEFAULT_PATH_FOR_NON_LISTED_NODES",
"paths":["/mnt/pv-data"]
}
]
}
Is this the secret you can get with kubectl get --namespace consul secrets/consul-bootstrap-acl-token --template={{.data.token}}
?
One thing I hit is that if you ever deployed consul service mesh, and didn't delete the namespace, the previous token will be left behind, but this would cause the cluster not to start. I had to delete the namespace to delete any secrets/configs left behind between different tests.
Will go ahead and close this issue. We've validated validated that this does work with manageSystemACLs = true on OpenShift 4.11 (with consul-k8s 1.0+). We have also addressed the pod security warnings you may encounter with ACLs enabled in Support running with restricted PSA enforcement enabled (part 1) by pglass · Pull Request #2572 · hashicorp/consul-k8s. Please open another issue if you run into any other issues on Consul K8s 1.0.x and above. We will plan to release a patch that includes that PR around mid August.
Community Note
Overview of the Issue
Currently using Consul Helm within Openshift 4.11.1 context (with Kubernetes 1.24). The automatic generation of system ACL fails
Reproduction Steps
When running helm install with the following
values.yml
:View error
Logs
Expected behavior
The system ACL with the secret Id is created und Kubernetes secret is build.