Bug with Kibana async_search permissions

[fenneh]

Running OSS ELK 7.8.0 using the OpenDistro security plugins.

Hit a pretty damning bug - when attempting to view any data in Kibana via the Discover pages you're hit with the following:

[security_exception] no permissions for [indices:data/read/async_search/submit] and User........

The permission isn't accessible within the Kibana UI, I've attempted to use the API to grant the permission but that also does not work (despite being visible in the UI)

PUT _opendistro/_security/api/roles/opendistro_security_anonymous
{
    "cluster_permissions" : [
      "indices:data/read/search",
      "indices:admin/mappings/get",
      "indices:admin/aliases/get",
      "indices:data/read/*"
    ],
    "index_permissions" : [
      {
        "index_patterns" : [
          "apm*",
          "logstash*",
          "filebeat-*",
        ],
        "dls" : "",
        "fls" : [ ],
        "masked_fields" : [ ],
        "allowed_actions" : [
          "crud",
          "create_index",
          "indices:data/read/async_search/submit"
        ]
      }
    ],
    "tenant_permissions" : [
      {
        "tenant_patterns" : [
          "DevOps"
        ],
        "allowed_actions" : [
          "kibana_all_write"
        ]
      },
      {
        "tenant_patterns" : [
          "global_tenant"
        ],
        "allowed_actions" : [
          "kibana_all_write"
        ]
      }
    ]
}

Async search was added in 7.7.0. This bug has been fixed by the SearchGuard guys - https://forum.search-guard.com/t/indices-data-read-async-search-submit-error/1842

Reported by another user on the OpenDistro forums; https://discuss.opendistrocommunity.dev/t/bug-indices-data-read-async-search-submit-error/3031

Is there any workarounds without granting users all_access to clusters?

[fenneh]

Any updates regarding this? This must be breaking for anyone using Elastic's Kibana with the OpenDistro Security plugin.

[SurgeVortex]

This is affecting my cluster too.

[SurgeVortex]

This is what the searchguard guys did to fix the issue. Can you please provide a similar fix? Override Kibana internal search

[Vhartmannfigaro]

Waiting for a bug correction i rollback to elastic search 7.6.1, Kibana 7.6.1 and open distro 1.7.0.0 plugin and with this configuration it works. _async_search is new in 7.7. Kibana_user was nos working so i created a custom rôle kibana_RO and that's my configuration:

In Internal_users.yml:

readonly_user:
  reserved: false

In roles_mapping.yml:
read_only_index:
  users:
  - readonly_user
kibana_RO:
  users:
  - readonly_user
kibana_read_only:
  users:
  - readonly_user

In roles.yml:

kibana_RO:
  reserved: true
  hidden: false
  description: Provide the minimum permissions for a kibana user
  cluster_permissions:
  - cluster_composite_ops
  index_permissions:
  - index_patterns:
    - ".kibana"
    - ".kibana-6"
    - ".kibana_*"
    allowed_actions:
    - read
    - delete
    - manage
    - index
  - index_patterns:
    - ".tasks"
    - ".management-beats"
    - "*:.tasks"
    - "*:.management-beats"
    allowed_actions:
    - indices_all
  tenant_permissions:
  - tenant_patterns:
    - global_tenant

In kibana.yml:

elasticsearch.requestHeadersWhitelist: ["securitytenant","Authorization"]
opendistro_security.multitenancy.enabled: false
opendistro_security.multitenancy.tenants.enable_global: true
opendistro_security.multitenancy.tenants.enable_private: true
opendistro_security.multitenancy.tenants.preferred: ["Private", "Global"]
opendistro_security.multitenancy.enable_filter: false

opendistro_security.readonly_mode.roles: [“kibana_read_only”]

[fenneh]

Waiting for a bug correction i rollback to elastic search 7.6.1, Kibana 7.6.1 and open distro 1.7.0.0 plugin and with this configuration it works. _async_search is new in 7.7. Kibana_user was nos working so i created a custom rôle kibana_RO and that's my configuration:

For anyone who's going to be attempting this, as far as I'm aware you can't roll elasticsearch back unless you've taken a snapshot to restore.

[rkbennett]

This can be worked around in 1.10.1.2 by manually adding the indices:data/read/async_search/submit and indices:data/read/async_search/get permissions via curl. Would be nice if these could be added to the list of selectable permissions, though

[shakomako-hibo]

hey rkbennett, iam using latest version of open distro security plugin + 7.9.1 kibana , can u elaborate on ur answer above. where do u add the permissions, to what role do u add them to

[ldrolez]

Same issue using the Opendistro stack v 7.8.0. @rkbennett on which indices these perms are missing?

[ldrolez]

I saw that SearchGuard workaround was to disable async search in Kibana, is there any progress on this?

[hkarpf]

Maybe this will be fixed in the upcoming version (1.13)?

https://github.com/opendistro-for-elasticsearch/community/issues/220

[massimolauri]

This is my workaround for my use case:

create action group: curl -XPUT "https://elasticsearch-host:9200/_opendistro/_security/api/actiongroups/async_fix_action_group" -H 'Content-Type: application/json' -d'{ "allowed_actions": [ "indices:data/read/async_search/submit", "indices:data/read/async_search/submit", "indices:data/read/async_search/delete", "indices:data/read/async_search/delete", "indices:data/read/async_search/get", "indices:data/read/async_search/get*" ]}'

Add to role action group.

[hkarpf]

This is my workaround for my use case:

create action group: curl -XPUT "https://elasticsearch-host:9200/_opendistro/_security/api/actiongroups/async_fix_action_group" -H 'Content-Type: application/json' -d'{ "allowed_actions": [ "indices:data/read/async_search/submit", "indices:data/read/async_search/submit", "indices:data/read/async_search/delete", "indices:data/read/async_search/delete", "indices:data/read/async_search/get", "indices:data/read/async_search/get*" ]}'

Add to role action group.

Tried this, but it still doesn't work if you only allow certain indices. Kibana tries to do: indices:data/read/async_search/submit on the allIndices=[*]. So if you don't give users permissions to all indices this doesn't work. :(

[francisco-hoo]

@hkarpf I think it's different things. I've installed and tested, and you still need to make sure your user have a role with index_pattens: ["*"] and allowed_action: ["indices:data/read/async_search/submit"] at least.

In some use cases it's insecure giving those permission that grant access to all indices... even if it's only indices:data/read/async_search/submit

[hkarpf]

Hopefully the new opensearch dashboards can take advantage of this feature.

[hkarpf]

Any word on if this is fixed in the latest distros of Opendistro?

[davidlago]

We are doing some "spring cleaning in the fall", and to make sure we focus our energies on the right issues and we get a better picture of the state of the repo, we are closing all issues that we are carrying over from the ODFE era (ODFE is no longer supported/maintained, see post here).

If you believe this issue should still be considered for current versions of OpenSearch, apologies! Please let us know by re-opening it.

Thanks!