search

opensearch-project / ansible-playbook

🤖 A community repository for Ansible Playbook of OpenSearch Project.
https://opensearch.org/
Apache License 2.0
86 stars 96 forks source link

[BUG][Security Plugin] Can't upload configuration #96

Closed ng-bsy closed 2 years ago

ng-bsy commented 2 years ago

Describe the bug

This tool will be deprecated in the next major release of OpenSearch **
** https://github.com/opensearch-project/security/issues/1755           **
**************************************************************************
Security Admin v7
Will connect to 10.10.1.77:9200 ... done
Connected as *****************"
OpenSearch Version: 2.3.0
Contacting opensearch cluster 'opensearch' and wait for YELLOW clusterstate ...
Clustername: *****
Clusterstate: GREEN
Number of nodes: 5
Number of data nodes: 5
.opendistro_security index already exists, so we do not need to create one.
Populate config from /usr/share/opensearch/plugins/opensearch-security/securityconfig/
ERR: Seems /usr/share/opensearch/plugins/opensearch-security/securityconfig/config.yml is not in OpenSearch Security 7 format: java.io.FileNotFoundException: /usr/share/opensearch/plugins/opensearch-security/securityconfig/config.yml (No such file or directory)
Will update '/roles' with /usr/share/opensearch/plugins/opensearch-security/securityconfig/roles.yml 
   SUCC: Configuration for 'roles' created or updated
Will update '/rolesmapping' with /usr/share/opensearch/plugins/opensearch-security/securityconfig/roles_mapping.yml 
   SUCC: Configuration for 'rolesmapping' created or updated
Will update '/internalusers' with /usr/share/opensearch/plugins/opensearch-security/securityconfig/internal_users.yml 
   SUCC: Configuration for 'internalusers' created or updated
ERR: Seems /usr/share/opensearch/plugins/opensearch-security/securityconfig/action_groups.yml is not in OpenSearch Security 7 format: java.io.FileNotFoundException: /usr/share/opensearch/plugins/opensearch-security/securityconfig/action_groups.yml (No such file or directory)
Will update '/tenants' with /usr/share/opensearch/plugins/opensearch-security/securityconfig/tenants.yml 
   SUCC: Configuration for 'tenants' created or updated
ERR: Seems /usr/share/opensearch/plugins/opensearch-security/securityconfig/nodes_dn.yml is not in OpenSearch Security 7 format: java.io.FileNotFoundException: /usr/share/opensearch/plugins/opensearch-security/securityconfig/nodes_dn.yml (No such file or directory)
ERR: Seems /usr/share/opensearch/plugins/opensearch-security/securityconfig/whitelist.yml is not in OpenSearch Security 7 format: java.io.FileNotFoundException: /usr/share/opensearch/plugins/opensearch-security/securityconfig/whitelist.yml (No such file or directory)
ERR: cannot upload configuration, see errors above

To Reproduce Steps to reproduce the behavior:

  1. Clone https://github.com/opensearch-project/ansible-playbook.git
  2. Customize Variables in inventories/opensearch/hosts and inventories/opensearch/group_vars/all/all.yml 2.1 Fix various other problems with the Playbook, already mentioned in the Issues, which hinder the Playbook getting this far
  3. Run ansible-playbook -i inventories/opensearch/hosts opensearch.yml --extra-vars "admin_password=Test@123 kibanaserver_password=Test@6789" --become
  4. Get stuck on Security Plugin configuration | Initialize the opensearch security index in opensearch

Expected behavior I expect the Playbook to run successfully and not into errors.

Playbook Name roles/linux/opensearch/tasks/security.yml

Host/Environment (please complete the following information):

ng-bsy commented 2 years ago

Can't Finally can edit my own issues it seems... Wanted to add, that it's a fresh installation of OpenSearch

ng-bsy commented 2 years ago

It seems the Playbook isn't supplying needed configuration files: /usr/share/opensearch/plugins/opensearch-security/securityconfig/nodes_dn.yml /usr/share/opensearch/plugins/opensearch-security/securityconfig/whitelist.yml

/usr/share/opensearch/plugins/opensearch-security/securityconfig/config.yml is also missing, but this seems by design, when not using auth_type 'oidc'

ng-bsy commented 2 years ago

Where does the securityconfig/config.yml come from, by the way, when not using auth_type 'oidc' ?

https://github.com/opensearch-project/ansible-playbook/blob/main/roles/linux/opensearch/tasks/security.yml#L149-L158

ng-bsy commented 2 years ago

This bug was introduced in https://github.com/opensearch-project/ansible-playbook/commit/436136df1ad464279cc69297e024c3dcd6e0b246

peterzhuamazon commented 2 years ago

@ssi444 ^ Would you mind provide some context to the issue @ng-bsy is facing due to the commit? Thanks!

ng-bsy commented 2 years ago

@peterzhuamazon @ssi444 I had to apply the following change (reverting a part of mentioned commit) for the playbook to work, when configured with internal authentication:

image

I suppose there should be be added some kind of "else" clause to the "when ... 'oidc'..." to handle both configuration cases (internal & oidc)

ssi444 commented 2 years ago

@peterzhuamazon @ng-bsy 

-cd {{os_sec_plugin_conf_path }} - applies all files
-f {{os_sec_plugin_conf_path }}/internal_users.yml - applies only 1 file

I checked on version 1.3, I haven't looked at version 2 yet. I'll see how the time is and I'll write.

ssi444 commented 2 years ago

Where does the securityconfig/config.yml come from, by the way, when not using auth_type 'oidc' ?

https://github.com/opensearch-project/ansible-playbook/blob/main/roles/linux/opensearch/tasks/security.yml#L149-L158

This file is in the OpenSearch package. With auth_type 'internal', this file is simply not overwritten and that's it

ssi444 commented 2 years ago

@peterzhuamazon @ng-bsy

In version 2.x, the security plugin files were moved from the {os_home}/plugins/opensearch-security/securityconfig directory to the {os_home}/config/opensearch-security directory. To work correctly, you need to change the value of the os_sec_plugin_conf_path variable in the inventories/opensearch/group_vars/all/all.yml file

I had this

os_home: /OS/opensearch
os_conf_dir: /OS/opensearch/config
os_plugin_bin_path: /OS/opensearch/bin/opensearch-plugin
os_sec_plugin_conf_path: /OS/opensearch/plugins/opensearch-security/securityconfig
os_sec_plugin_tools_path: /OS/opensearch/plugins/opensearch-security/tools

os_dashboards_home: /OS/opensearch-dashboards
os_dashboards_conf_dir: /OS/opensearch-dashboards/config
os_dashboards_plugin_bin_path: /OS/opensearch-dashboards/bin/opensearch-dashboards-plugin

It became so

os_home: /OS/opensearch
os_conf_dir: /OS/opensearch/config
os_plugin_bin_path: /OS/opensearch/bin/opensearch-plugin
os_sec_plugin_conf_path: /OS/opensearch/config/opensearch-security
os_sec_plugin_tools_path: /OS/opensearch/plugins/opensearch-security/tools

os_dashboards_home: /OS/opensearch-dashboards
os_dashboards_conf_dir: /OS/opensearch-dashboards/config
os_dashboards_plugin_bin_path: /OS/opensearch-dashboards/bin/opensearch-dashboards-plugin
ssi444 commented 2 years ago

Thus, I made only 3 changes to my configuration in order to deploy a cluster of version 2.3.0 instead of a cluster of version 1.3.2

  1. os_version: "1.3.2" -> "2.3.0"
  2. os_dashboards_version: "1.3.2" -> "2.3.0"
  3. os_sec_plugin_conf_path: "/OS/opensearch/plugins/opensearch-security/securityconfig" -> "/OS/opensearch/configuration/opensearch-security"
ssi444 commented 2 years ago

There is another pitfall associated with --become. This task will also be performed with elevated privileges, but on the local computer and if this cannot be done, an error will occur. I started connecting to the OS nodes by the root user and there was no problem, but if another user connects, for example, ubuntu, an error will occur. If you add become: no to this task, then it does not help. We need to think about it...

- name: Security Plugin configuration | Check that the files/internal_users.yml exists
  stat:
    path: files/internal_users.yml
  register: custom_users_result
  delegate_to: localhost
  run_once: true

and

- name: Security Plugin configuration | Check for a custom configuration for internal users and hash passwords for them
ng-bsy commented 2 years ago

3. os_sec_plugin_conf_path: "/OS/opensearch/plugins/opensearch-security/securityconfig" -> "/OS/opensearch/configuration/opensearch-security"

@ssi444 This helped a lot, thanks!

Edit: Although I edited it in roles/linux/opensearch/defaults/main.yml

peterzhuamazon commented 2 years ago

Fixed in #80