elastic / logstash

Logstash - transport and process your logs, events, or other data
https://www.elastic.co/products/logstash
Other
61 stars 3.5k forks source link

Document permissions and/or role required for user to run X-Pack features #9146

Closed dedemorton closed 2 weeks ago

dedemorton commented 6 years ago

Currently we document how to set up users for Logstash (here), but we don't document which roles are required when using specific features, like the pipeline viewer and monitoring.

(One exception is centralized config management, where we do document the required roles: https://www.elastic.co/guide/en/logstash/master/logstash-centralized-pipeline-management.html)

@karenzone You'll need to investigate which roles are required for using the pipeline viewer and monitoring.

jakelandis commented 6 years ago

I'm going to hijack this issue for my opinion on the current state of https://www.elastic.co/guide/en/logstash/current/ls-security.html and suggestions for how to fix.

Logstash Internal

 _xpack/security/role/logstash_writer -> ["manage_index_templates", "monitor"]

_xpack/security/user/logstash_internal ->  "roles" : [ "logstash_writer"]

input {
  elasticsearch {
    ...
    user => logstash_internal

Logstash User

_xpack/security/role/logstash_reader ->  ["read","view_index_metadata"]

_xpack/security/user/logstash_user -> [ "logstash_reader", "logstash_admin"],

Logstash System

_xpack/security/user/logstash_system/_password

xpack.monitoring.elasticsearch.username: logstash_system

_xpack/security/user/logstash_system/_enable

Logstash admin user

xpack.management.elasticsearch.username: logstash_admin_user

Suggested fixes:

To discuss ...

jakelandis commented 6 years ago

We also need to reconcile any changes/documentation with this: https://www.elastic.co/guide/en/x-pack/current/built-in-roles.html

dedemorton commented 6 years ago

Looping @lcawl in on this discussion since it pertains to security. TBH, I'm not sure who wrote the original text here because the content has moved around a lot and lost its history. She might have more detail on specific users/roles and why we suggest specific privileges.

I did test this content last year. If you change the user names/roles/privileges, you'll need to make sure you update (and test) any topics that refer to them.

lcawl commented 6 years ago

The original home of that page was https://www.elastic.co/guide/en/x-pack/5.0/logstash.html, but it hasn't had too many substantive updates since that time, so there's definitely room for improvement.

With respect to the logstash_system built-in user, it's one of the ones that's created when you have X-Pack security enabled and its initial password is set by using the setup-password command, per https://www.elastic.co/guide/en/x-pack/master/setting-up-authentication.html#built-in-users I believe the statement "By default, the logstash_system user does not have a password" in https://www.elastic.co/guide/en/logstash/master/ls-security.html is incorrect, so that will need to be corrected too.

I'm happy to help with these updates.

jakelandis commented 6 years ago

I removed the suggested fixes as they were too aggressive w/r/t passivity. We can discuss and I will take another stab at a suggestion.

jakelandis commented 6 years ago

There are basically three index patterns needed for Logstash default.

For logstash* , there are primary two personas:

For .logstash*, there are two personas:

For .monitoring-logstash*

@ycombinator @andrewvc - would you agree that these are personas that should back suggestions for RBAC ?

jakelandis commented 6 years ago

Out of the box, it appears that we have 2 pre-defined roles (and 0 pre-defined users):

Logstash admin role:

image

and Logstash system role:

image

It seems that the Logstash admin role is target at "the Human changing the central config " , which seems correct.

The documentation states that Logstash system role "Grants access necessary for the Logstash system user to send system-level data (such as monitoring) to Elasticsearch.' , which seems to be targeting the "the Logstash system writing the monitoring data, and reading the management data" persona. However, the privileges don't appear be correct since there are no index level privs and the existing documentation refers to enabling a user that does not work. I think persona needs the most attention, and may require changes to the default roles.

The other personas seem to be documented as logstash reader/writer roles which also seem correct, but may benefit from some updates for clarification.

jakelandis commented 6 years ago

I think this proposal covers the personas and is a minimal difference to what is currently there.

Human / System User Role
System logstash_system_output logstash_system (for cluster health)+ logstash_writer (for logstash)
System logstash_system logstash_system (for cluster health) + logstash_admin (for .logstash) + ?? role (for .monitoring-logstash)
Human logastash_user logstash_reader (for logstash)
Human logstash_user_admin logstash_reader (for logstash) + logstash_admin (for .logstash) + monitoring_user (for .monitoring-logstash)
jakelandis commented 6 years ago
bin/x-pack/setup-passwords interactive

^^ is needed to bootstrap the logstash_system user.

logstash_system The user Logstash uses when storing monitoring information in Elasticsearch.

so I think that bootstrapping step is needed to give the .monitoring-logstash* write privs ... but it is unclear how to do that for cloud deployment.

jakelandis commented 6 years ago

?? role (for .monitoring-logstash)

I confirmed that the logstash_system role has implicit privs to write to the monitoring index.

Per the proposal above I suggest we should (by default) give the logstash_system user the logstash_admin role to as well to allow (by default) the logstash_system user to be used for both monitoring and management.

is needed to bootstrap the logstash_system user.

Also confirmed that this user is not currently available via cloud, so we will likely want to explicitly document how to set this user up. (note the role is present , just not the user)

ycombinator commented 6 years ago

For .logstash*, there are two personas:

the Logstash system reading the management data the Human changing the central config (management data)

I agree with the personas.

(note we don't to support reading the central config as standalone persona due to security reasons)

Can you elaborate on this? I've always wondered about this: why the Logstash system doesn't get just read access to .logstash, because it doesn't need to write to it. From a security POV, I'd think we'd want to provide as restrictive access as necessary, no?

jakelandis commented 6 years ago

Can you elaborate on this? I've always wondered about this: why the Logstash system doesn't get just read access to .logstash, because it doesn't need to write to it

The comment was in reference to the human use case, where we should never give a human the ability to read but not write. Doing that would open the door to accidentally give a non-admin user the ability see potentially sensitive information.

I wasn't part of the part the decision w/r/t the definition of the role, but it does make sense for the human use case.

From a security POV, I'd think we'd want to provide as restrictive access as necessary, no?

Agreed for the system use case. Perhaps we should try address that with a change to the logstash_system role to provide that role with the read access .logstash ? _(that would mean in a minor change to the proposed logstash_systemoutput user above)

ycombinator commented 6 years ago

The comment was in reference to the human use case, where we should never give a human the ability to read but not write. Doing that would open the door to accidentally give a non-admin user the ability see potentially sensitive information.

The comment was in reference to the human use case, where we should never give a human the ability to read but not write. Doing that would open the door to accidentally give a non-admin user the ability see potentially sensitive information.

I wasn't part of the part the decision w/r/t the definition of the role, but it does make sense for the human use case.

Agreed. Makes sense. Thanks for the explanation.

Perhaps we should try address that with a change to the logstash_system role to provide that role with the read access .logstash ? _(that would mean in a minor change to the proposed logstash_systemoutput user above)

Yeah, this makes sense to me. It does mean that all existing users of Logstash would've given their Logstash (system) read and write privileges to .logstash but at least new users would have the option of using a more restricted user (logstash_system_output).

dedemorton commented 6 years ago

For the Beats -> LS -> ES (with secured cluster) use case, we also need to document the privileges that are required to write/read to beats indexes for users who want to use the beats dashboards--that is, users who have this config:

output {
  elasticsearch {
    hosts => "localhost:9200"
    manage_template => false
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" 
    document_type => "%{[@metadata][type]}"
    user => "logstash_internal"
    password => "x-pack-test-password"
  }
}

You get the following security exception if the user doesn't have the correct privileges:

[2018-05-17T18:35:26,780][INFO ][logstash.outputs.elasticsearch] retrying failed action with 
response code: 403 ({"type"=>"security_exception", "reason"=>"action [indices:admin/create] is 
unauthorized for user [logstash_internal]"})
ashokbellur commented 6 years ago

Hello, I am exactly running into the above issue using stack 6.3.1.

I have a secured cluster with almost similar config.

Is this issue resolved now? ][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"security_exception", "reason"=>"action [indices:admin/create] is unauthorized for user [logstash_internal]"})

dedemorton commented 6 years ago

@ashokbellur Make sure your logasth_internal user has write and create_index privileges on the Beats index you are trying to send events to. Note that this GitHub issue is for tracking updates required to the documentation. If you have questions about configuring Logstash and/or Beats, please ask in the discussion forums. See:

https://discuss.elastic.co/c/logstash https://discuss.elastic.co/c/beats

dedemorton commented 2 years ago

@karenzone Is this issue still valid, or should I close it?

dedemorton commented 2 weeks ago

closing because this issue is soooo old.