[Enhancement]opensearch.yml permission denied during startup

sdwerwed commented 2 years ago

Chart version: 1.0.5 Image version: 1.2.4

Permission error for opensearch.yml because user is root (with read write), group is opensearch (with read)

To fix this I had to:

apply the helm chart
Change the spec of statefulset

from:

volumes:
  - configMap:
      defaultMode: 420
      name: opensearch-cluster-master-config
    name: config

to:

volumes:
  - configMap:
      defaultMode: 0660
      name: opensearch-cluster-master-config
    name: config

wait the rollout restart

Request: Add in values.yml option to set the mode

peterzhuamazon commented 2 years ago

This @sdwerwed for the suggestion, is it possible that you can help contribute this feature to the charts? Thanks.

sdwerwed commented 2 years ago

Hello, I could contribute that but it will not be very soon, if anyone else from community can implement that feel free to proceed.

kyrofa commented 8 months ago

I'm confused, I'm seeing this as well, how is this not considered a bug? Doesn't it mean that opensearch isn't configuring itself properly?

prudhvigodithi commented 7 months ago

Hey @sdwerwed this should be fixed in chart version 2.17.3, can you please try with 2.17.3? Thanks @kyrofa @bbarani @peterzhuamazon

prudhvigodithi commented 7 months ago

Closing this issue, please re-open if required. Thanks

alexmorbo commented 7 months ago

@prudhvigodithi Hi! Maybe you forgot to release new version to repository?

Error: could not download chart: chart "opensearch" version "2.17.3" not found in https://opensearch-project.github.io/helm-charts repository

prudhvigodithi commented 7 months ago

Hey @alexmorbo somehow the GH action failed, can you please try again? @bbarani @peterzhuamazon

alexmorbo commented 7 months ago

@prudhvigodithi thanks, chart available now, but still some problems.

I have config value:

config:
  opensearch.yml:
    cluster.name: opensearch-cluster
    network.host: 0.0.0.0

but in pod i have this:

kubectl -n opensearch exec opensearch-0 -- cat /usr/share/opensearch/config/opensearch.yml
Defaulted container "opensearch" out of: opensearch, fsgroup-volume (init), configfile (init)
cluster.name: opensearch-cluster
network.host: 0.0.0.0

######## Start OpenSearch Security Demo Configuration ########
# WARNING: revise all the lines below before you go into production
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:
  - CN=kirk,OU=client,O=client,L=test, C=de

plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [".plugins-ml-config", ".plugins-ml-connector", ".plugins-ml-model-group", ".plugins-ml-model", ".plugins-ml-task", ".plugins-ml-conversation-meta", ".plugins-ml-conversation-interactions", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".ql-datasources", ".opendistro-asynchronous-search-response*", ".replication-metadata-store", ".opensearch-knn-models", ".geospatial-ip2geo-data*"]
node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ########

I think opensearch.yml file is default and not takes data from values

prudhvigodithi commented 7 months ago

@alexmorbo Can you try as https://github.com/opensearch-project/helm-charts/blob/main/charts/opensearch/values.yaml#L49

config:
  opensearch.yml: | 
    cluster.name: opensearch-cluster
    network.host: 0.0.0.0

alexmorbo commented 7 months ago

@prudhvigodithi not works

kubectl -n opensearch get configmap opensearch-config -o yaml
apiVersion: v1
data:
  opensearch.yml: |
    cluster.name: opensearch-cluster
    network.host: 0.0.0.0
kind: ConfigMap
metadata:
....

and

kubectl -n opensearch exec opensearch-0 -- cat /usr/share/opensearch/config/opensearch.yml
Defaulted container "opensearch" out of: opensearch, fsgroup-volume (init), configfile (init)
cluster.name: opensearch-cluster
network.host: 0.0.0.0

######## Start OpenSearch Security Demo Configuration ########
# WARNING: revise all the lines below before you go into production
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:
  - CN=kirk,OU=client,O=client,L=test, C=de

plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [".plugins-ml-config", ".plugins-ml-connector", ".plugins-ml-model-group", ".plugins-ml-model", ".plugins-ml-task", ".plugins-ml-conversation-meta", ".plugins-ml-conversation-interactions", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".ql-datasources", ".opendistro-asynchronous-search-response*", ".replication-metadata-store", ".opensearch-knn-models", ".geospatial-ip2geo-data*"]
node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ########

prudhvigodithi commented 7 months ago

Hey @alexmorbo this should work.

My config settings in values file

config:
  # Values must be YAML literal style scalar / YAML multiline string.
  # <filename>: |
  #   <formatted-value(s)>
  # log4j2.properties: |
  #   status = error
  #
  #   appender.console.type = Console
  #   appender.console.name = console
  #   appender.console.layout.type = PatternLayout
  #   appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
  #
  #   rootLogger.level = info
  #   rootLogger.appenderRef.console.ref = console
  opensearch.yml: |
    cluster.name: opensearch-cluster

    # Bind to all interfaces because we don't know what IP address Docker will assign to us.
    network.host: 0.0.0.0

    # Setting network.host to a non-loopback address enables the annoying bootstrap checks. "Single-node" mode disables them again.
    # Implicitly done if ".singleNode" is set to "true".
    # discovery.type: single-node

    # Start OpenSearch Security Demo Configuration
    # WARNING: revise all the lines below before you go into production
    plugins:
      security:
        ssl:
          transport:
            pemcert_filepath: esnode.pem
            pemkey_filepath: esnode-key.pem
            pemtrustedcas_filepath: root-ca.pem
            enforce_hostname_verification: false
          http:
            enabled: true
            pemcert_filepath: esnode.pem
            pemkey_filepath: esnode-key.pem
            pemtrustedcas_filepath: root-ca.pem
        allow_unsafe_democertificates: true
        allow_default_init_securityindex: true
        authcz:
          admin_dn:
            - CN=kirk,OU=client,O=client,L=test,C=de
        audit.type: internal_opensearch
        enable_snapshot_restore_privilege: true
        check_snapshot_restore_write_privileges: true
        restapi:
          roles_enabled: ["all_access", "security_rest_api_access"]
        system_indices:
          enabled: true
          indices:
            [
              ".opendistro-alerting-config",
              ".opendistro-alerting-alert*",
              ".opendistro-anomaly-results*",
              ".opendistro-anomaly-detector*",
              ".opendistro-anomaly-checkpoints",
              ".opendistro-anomaly-detection-state",
              ".opendistro-reports-*",
              ".opendistro-notifications-*",
              ".opendistro-notebooks",
              ".opendistro-asynchronous-search-response*",
            ]
    ######## End OpenSearch Security Demo Configuration ########
  # log4j2.properties:

The `config/opensearch.yml` inside the pod

cluster.name: opensearch-cluster

# Bind to all interfaces because we don't know what IP address Docker will assign to us.
network.host: 0.0.0.0

# Setting network.host to a non-loopback address enables the annoying bootstrap checks. "Single-node" mode disables them again.
# Implicitly done if ".singleNode" is set to "true".
# discovery.type: single-node

# Start OpenSearch Security Demo Configuration
# WARNING: revise all the lines below before you go into production
plugins:
  security:
    ssl:
      transport:
        pemcert_filepath: esnode.pem
        pemkey_filepath: esnode-key.pem
        pemtrustedcas_filepath: root-ca.pem
        enforce_hostname_verification: false
      http:
        enabled: true
        pemcert_filepath: esnode.pem
        pemkey_filepath: esnode-key.pem
        pemtrustedcas_filepath: root-ca.pem
    allow_unsafe_democertificates: true
    allow_default_init_securityindex: true
    authcz:
      admin_dn:
        - CN=kirk,OU=client,O=client,L=test,C=de
    audit.type: internal_opensearch
    enable_snapshot_restore_privilege: true
    check_snapshot_restore_write_privileges: true
    restapi:
      roles_enabled: ["all_access", "security_rest_api_access"]
    system_indices:
      enabled: true
      indices:
        [
          ".opendistro-alerting-config",
          ".opendistro-alerting-alert*",
          ".opendistro-anomaly-results*",
          ".opendistro-anomaly-detector*",
          ".opendistro-anomaly-checkpoints",
          ".opendistro-anomaly-detection-state",
          ".opendistro-reports-*",
          ".opendistro-notifications-*",
          ".opendistro-notebooks",
          ".opendistro-asynchronous-search-response*",
        ]
######## End OpenSearch Security Demo Configuration ########

######## Start OpenSearch Security Demo Configuration ########
# WARNING: revise all the lines below before you go into production
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:
  - CN=kirk,OU=client,O=client,L=test, C=de

plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [".plugins-ml-config", ".plugins-ml-connector", ".plugins-ml-model-group", ".plugins-ml-model", ".plugins-ml-task", ".plugins-ml-conversation-meta", ".plugins-ml-conversation-interactions", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".ql-datasources", ".opendistro-asynchronous-search-response*", ".replication-metadata-store", ".opensearch-knn-models", ".geospatial-ip2geo-data*"]
node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ########

Those are the defaults added by the security plugin, try updating the the cluster.name to opensearch-cluster-test it should get updated and the rest of the lines are added by the security plugin when ran the demo configuration.

config:
  opensearch.yml: | 
    cluster.name: opensearch-cluster-test
    network.host: 0.0.0.0

cluster.name: opensearch-cluster-test
network.host: 0.0.0.0

######## Start OpenSearch Security Demo Configuration ########
# WARNING: revise all the lines below before you go into production
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false
plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem
plugins.security.allow_unsafe_democertificates: true
plugins.security.allow_default_init_securityindex: true
plugins.security.authcz.admin_dn:
  - CN=kirk,OU=client,O=client,L=test, C=de

plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privileges: true
plugins.security.restapi.roles_enabled: ["all_access", "security_rest_api_access"]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: [".plugins-ml-config", ".plugins-ml-connector", ".plugins-ml-model-group", ".plugins-ml-model", ".plugins-ml-task", ".plugins-ml-conversation-meta", ".plugins-ml-conversation-interactions", ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opensearch-notifications-*", ".opensearch-notebooks", ".opensearch-observability", ".ql-datasources", ".opendistro-asynchronous-search-response*", ".replication-metadata-store", ".opensearch-knn-models", ".geospatial-ip2geo-data*"]
node.max_local_storage_nodes: 3
######## End OpenSearch Security Demo Configuration ########

bdkuhman commented 3 months ago

Hey @alexmorbo this should work.

My config settings in values file
config:
 <TRUNCATED>

Edit: I found this line, which by adding a comment to the config, stops the duplicate config from being generated.

Hey @prudhvigodithi,

I'm trying to overwrite the configuration and set tls certs, but I'm seeing the demo config getting duplicated.

values.yml

```yaml config: opensearch.yml: | #test cluster.name: opensearch-cluster # Bind to all interfaces because we don't know what IP address Docker will assign to us. network.host: 0.0.0.0 # Setting network.host to a non-loopback address enables the annoying bootstrap checks. "Single-node" mode disables them again. # Implicitly done if ".singleNode" is set to "true". # discovery.type: single-node # Start OpenSearch Security Demo Configuration # WARNING: revise all the lines below before you go into production plugins: security: ssl: transport: pemcert_filepath: /certs/tls.crt pemkey_filepath: /certs/tls.key pemtrustedcas_filepath: /certs/ca.crt enforce_hostname_verification: false http: enabled: true pemcert_filepath: /certs/tls.crt pemkey_filepath: /certs/tls.key pemtrustedcas_filepath: /certs/ca.crt allow_unsafe_democertificates: true allow_default_init_securityindex: true authcz: admin_dn: - CN=kirk,OU=client,O=client,L=test,C=de audit.type: internal_opensearch enable_snapshot_restore_privilege: true check_snapshot_restore_write_privileges: true restapi: roles_enabled: ["all_access", "security_rest_api_access"] system_indices: enabled: true indices: [ ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opendistro-notifications-*", ".opendistro-notebooks", ".opendistro-asynchronous-search-response*", ] ######## End OpenSearch Security Demo Configuration ######## #alsotest ```

results in producing configmap:

```yaml #test cluster.name: opensearch-cluster # Bind to all interfaces because we don't know what IP address Docker will assign to us. network.host: 0.0.0.0 # Setting network.host to a non-loopback address enables the annoying bootstrap checks. "Single-node" mode disables them again. # Implicitly done if ".singleNode" is set to "true". # discovery.type: single-node # Start OpenSearch Security Demo Configuration # WARNING: revise all the lines below before you go into production plugins: security: ssl: transport: pemcert_filepath: /certs/tls.crt pemkey_filepath: /certs/tls.key pemtrustedcas_filepath: /certs/ca.crt enforce_hostname_verification: false http: enabled: true pemcert_filepath: /certs/tls.crt pemkey_filepath: /certs/tls.key pemtrustedcas_filepath: /certs/ca.crt allow_unsafe_democertificates: true allow_default_init_securityindex: true authcz: admin_dn: - CN=kirk,OU=client,O=client,L=test,C=de audit.type: internal_opensearch enable_snapshot_restore_privilege: true check_snapshot_restore_write_privileges: true restapi: roles_enabled: ["all_access", "security_rest_api_access"] system_indices: enabled: true indices: [ ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opendistro-notifications-*", ".opendistro-notebooks", ".opendistro-asynchronous-search-response*", ] ######## End OpenSearch Security Demo Configuration ######## #alsotest ```

and in the pod:

```terminal [opensearch@opensearch-cluster-master-2 ~]$ cat config/opensearch.yml ``` ```yaml #test cluster.name: opensearch-cluster # Bind to all interfaces because we don't know what IP address Docker will assign to us. network.host: 0.0.0.0 # Setting network.host to a non-loopback address enables the annoying bootstrap checks. "Single-node" mode disables them again. # Implicitly done if ".singleNode" is set to "true". # discovery.type: single-node # Start OpenSearch Security Demo Configuration # WARNING: revise all the lines below before you go into production plugins: security: ssl: transport: pemcert_filepath: /certs/tls.crt pemkey_filepath: /certs/tls.key pemtrustedcas_filepath: /certs/ca.crt enforce_hostname_verification: false http: enabled: true pemcert_filepath: /certs/tls.crt pemkey_filepath: /certs/tls.key pemtrustedcas_filepath: /certs/ca.crt allow_unsafe_democertificates: true allow_default_init_securityindex: true authcz: admin_dn: - CN=kirk,OU=client,O=client,L=test,C=de audit.type: internal_opensearch enable_snapshot_restore_privilege: true check_snapshot_restore_write_privileges: true restapi: roles_enabled: ["all_access", "security_rest_api_access"] system_indices: enabled: true indices: [ ".opendistro-alerting-config", ".opendistro-alerting-alert*", ".opendistro-anomaly-results*", ".opendistro-anomaly-detector*", ".opendistro-anomaly-checkpoints", ".opendistro-anomaly-detection-state", ".opendistro-reports-*", ".opendistro-notifications-*", ".opendistro-notebooks", ".opendistro-asynchronous-search-response*", ] ######## End OpenSearch Security Demo Configuration ######## #alsotest ######## Start OpenSearch Security Demo Configuration ######## # WARNING: revise all the lines below before you go into production plugins.security.ssl.transport.pemcert_filepath: esnode.pem plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem plugins.security.ssl.transport.enforce_hostname_verification: false plugins.security.ssl.http.enabled: true plugins.security.ssl.http.pemcert_filepath: esnode.pem plugins.security.ssl.http.pemkey_filepath: esnode-key.pem plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem plugins.security.allow_unsafe_democertificates: true plugins.security.allow_default_init_securityindex: true plugins.security.authcz.admin_dn: ['CN=kirk,OU=client,O=client,L=test,C=de'] plugins.security.audit.type: internal_opensearch plugins.security.enable_snapshot_restore_privilege: true plugins.security.check_snapshot_restore_write_privileges: true plugins.security.restapi.roles_enabled: [all_access, security_rest_api_access] plugins.security.system_indices.enabled: true plugins.security.system_indices.indices: [.plugins-ml-agent, .plugins-ml-config, .plugins-ml-connector, .plugins-ml-controller, .plugins-ml-model-group, .plugins-ml-model, .plugins-ml-task, .plugins-ml-conversation-meta, .plugins-ml-conversation-interactions, .plugins-ml-memory-meta, .plugins-ml-memory-message, .plugins-ml-stop-words, .opendistro-alerting-config, .opendistro-alerting-alert*, .opendistro-anomaly-results*, .opendistro-anomaly-detector*, .opendistro-anomaly-checkpoints, .opendistro-anomaly-detection-state, .opendistro-reports-*, .opensearch-notifications-*, .opensearch-notebooks, .opensearch-observability, .ql-datasources, .opendistro-asynchronous-search-response*, .replication-metadata-store, .opensearch-knn-models, .geospatial-ip2geo-data*, .plugins-flow-framework-config, .plugins-flow-framework-templates, .plugins-flow-framework-state] node.max_local_storage_nodes: 3 ######## End OpenSearch Security Demo Configuration ######## ```

opensearch-project / helm-charts