Open danielkubat opened 3 months ago
I think that once/if https://github.com/opensearch-project/opensearch-k8s-operator/pull/754 is merged, this problem will be solved.
[Triage]
Hey @danielkubat and @ebenezar-mccoy, based on my testing in past https://github.com/opensearch-project/opensearch-k8s-operator/issues/703#issuecomment-1942311512, change this with Custom Admin User should update the admin user password, else should remain to password as admin
.
Thank you
Adding @bbarani @salyh @jochenkressin @pchmielnik @bbarani
I tried to create a cluster with the following config but it's not working with the same error
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: generate-secrets
namespace: opensearch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: create-secret-role
namespace: opensearch
rules:
- verbs: ["create", "get"]
apiGroups: [""]
resources: ["secrets"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: generate-secrets-create-secret
namespace: opensearch
subjects:
- kind: ServiceAccount
name: generate-secrets
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: create-secret-role
---
apiVersion: batch/v1
kind: Job
metadata:
name: generate-secrets
namespace: opensearch
spec:
template:
spec:
serviceAccountName: generate-secrets
securityContext:
runAsUser: 0
runAsGroup: 0
containers:
- name: kubectl
image: docker.io/bitnami/kubectl:1.29
command: [/bin/sh, -c]
args:
- |
apt update
apt install -y python3 python3-bcrypt
PASSWORD=$(tr -dc 'A-Za-z0-9' < /dev/urandom | head -c 40)
PASSWORD_HASH=$(cat <<EOF | python3
import bcrypt
print(
bcrypt.hashpw(
"$PASSWORD".encode("utf-8"),
bcrypt.gensalt(12, prefix=b"2a"),
).decode("utf-8")
)
EOF
)
cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: admin-credentials
namespace: opensearch
data:
username: $(printf "admin" | base64 -w 0)
password: $(printf "$PASSWORD" | base64 -w 0)
EOF
cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: security-config
namespace: opensearch
stringData:
internal_users.yml: |-
_meta:
type: "internalusers"
config_version: 2
admin:
hash: $PASSWORD_HASH
reserved: true
backend_roles:
- "admin"
description: "Admin user"
EOF
exit 0
restartPolicy: OnFailure
---
apiVersion: opensearch.opster.io/v1
kind: OpenSearchCluster
metadata:
name: default-cluster
namespace: opensearch
spec:
general:
version: "2.14.0"
httpPort: 9200
vendor: opensearch
serviceName: opensearch
monitoring:
enable: true
pluginsList: ["repository-s3"]
setVMMaxMapCount: true
security:
config:
adminCredentialsSecret:
name: admin-credentials
securityConfigSecret:
name: security-config
dashboards:
version: "2.14.0"
enable: true
replicas: 1
resources:
requests:
memory: 256Mi
cpu: 50m
limits:
memory: 1Gi
cpu: 500m
confMgmt:
smartScaler: true
nodePools:
- component: masters
replicas: 3
diskSize: 4Gi
nodeSelector:
resources:
requests:
memory: 512Mi
cpu: 50m
limits:
memory: 768Mi
cpu: 250m
roles:
- master
- ingest
- component: nodes
replicas: 3
diskSize: 40Gi
nodeSelector:
resources:
requests:
memory: 1Gi
cpu: 100m
limits:
memory: 2Gi
cpu: 1000m
roles:
- data
This should run a Job
to follow steps in Custom Admin User but bootstrap
still crashes with No custom admin password found. Please provide a password via the environment variable OPENSEARCH_INITIAL_ADMIN_PASSWORD.
It works adding these env vars. The operator can add this var referencing the Secret its stored in with ValueFrom. I'll put up a PR tonight after work if I have time
I opened https://github.com/opensearch-project/opensearch-k8s-operator/pull/816. It may need some input/review from contributors/maintainers and additional changes.
Couldn't get it working locally without this but the current code consistency works in CI so there must be something else missing. I'm thinking something is triggering the security plugin & security demo script some times
Hmm I can't seem to repro on Github Actions even with the config that was causing me errors locally. Maybe there's some kind of race condition or something at play
probably you can add
bootstrap:
additionalConfig:
OPENSEARCH_INITIAL_ADMIN_PASSWORD: 'password'
and it should work, cause operator merge this values to pod env
probably you can add
bootstrap: additionalConfig: OPENSEARCH_INITIAL_ADMIN_PASSWORD: 'password'
and it should work, cause operator merge this values to pod env
I think it's still missing from one of the pods--maybe bootstrap pod
Edit: Yeah it looks like that's static env vars for bootstrap pod https://github.com/opensearch-project/opensearch-k8s-operator/blob/main/opensearch-operator/pkg/builders/cluster.go#L470
That was one of the things I adjusted in the PR (make sure all pods use common env vars)
What is the bug?
Unable to perform new deployment of opensearch
2.12.0
using opensearch-operator2.5.1
.How can one reproduce the bug?
Create clean opensearch
2.12.0
cluster using opensearch-operator2.5.1
.What is the expected behavior?
Cluster is up and running including
bootstrap
pod.What is your host/environment?
GKE (
1.27.8-gke.1067004
)Do you have any additional context?
Opensearch
2.12.0
explicitely requires to setOPENSEARCH_INITIAL_ADMIN_PASSWORD
environment variable for each pod. This is possible fornodePools
but not forbootstrap
pod, as the helm manifest only supportsadditionalConfig
, therefore pod is crashing.Setting up Custom Admin User does not help, as pod strictly requires
OPENSEARCH_INITIAL_ADMIN_PASSWORD
defined as the variable.