Closed dedemorton closed 1 month ago
I'm going to hijack this issue for my opinion on the current state of https://www.elastic.co/guide/en/logstash/current/ls-security.html and suggestions for how to fix.
_xpack/security/role/logstash_writer -> ["manage_index_templates", "monitor"]
_xpack/security/user/logstash_internal -> "roles" : [ "logstash_writer"]
input {
elasticsearch {
...
user => logstash_internal
_xpack/security/role/logstash_reader -> ["read","view_index_metadata"]
_xpack/security/user/logstash_user -> [ "logstash_reader", "logstash_admin"],
_xpack/security/user/logstash_system/_password
xpack.monitoring.elasticsearch.username: logstash_system
_xpack/security/user/logstash_system/_enable
xpack.management.elasticsearch.username: logstash_admin_user
To discuss ...
We also need to reconcile any changes/documentation with this: https://www.elastic.co/guide/en/x-pack/current/built-in-roles.html
Looping @lcawl in on this discussion since it pertains to security. TBH, I'm not sure who wrote the original text here because the content has moved around a lot and lost its history. She might have more detail on specific users/roles and why we suggest specific privileges.
I did test this content last year. If you change the user names/roles/privileges, you'll need to make sure you update (and test) any topics that refer to them.
The original home of that page was https://www.elastic.co/guide/en/x-pack/5.0/logstash.html, but it hasn't had too many substantive updates since that time, so there's definitely room for improvement.
With respect to the logstash_system built-in user, it's one of the ones that's created when you have X-Pack security enabled and its initial password is set by using the setup-password command, per https://www.elastic.co/guide/en/x-pack/master/setting-up-authentication.html#built-in-users I believe the statement "By default, the logstash_system user does not have a password" in https://www.elastic.co/guide/en/logstash/master/ls-security.html is incorrect, so that will need to be corrected too.
I'm happy to help with these updates.
I removed the suggested fixes as they were too aggressive w/r/t passivity. We can discuss and I will take another stab at a suggestion.
There are basically three index patterns needed for Logstash default.
logstash*
- the events ingested.logstash*
- the data that backs management (x-pack) .monitoring-logstash*
- the data that backs monitoring (x-pack)For logstash*
, there are primary two personas:
For .logstash*
, there are two personas:
For .monitoring-logstash*
@ycombinator @andrewvc - would you agree that these are personas that should back suggestions for RBAC ?
Out of the box, it appears that we have 2 pre-defined roles (and 0 pre-defined users):
Logstash admin role:
and Logstash system role:
It seems that the Logstash admin role is target at "the Human changing the central config " , which seems correct.
The documentation states that Logstash system role "Grants access necessary for the Logstash system user to send system-level data (such as monitoring) to Elasticsearch.' , which seems to be targeting the "the Logstash system writing the monitoring data, and reading the management data" persona. However, the privileges don't appear be correct since there are no index level privs and the existing documentation refers to enabling a user that does not work. I think persona needs the most attention, and may require changes to the default roles.
The other personas seem to be documented as logstash reader/writer roles which also seem correct, but may benefit from some updates for clarification.
I think this proposal covers the personas and is a minimal difference to what is currently there.
Human / System | User | Role |
---|---|---|
System | logstash_system_output | logstash_system (for cluster health)+ logstash_writer (for logstash ) |
System | logstash_system | logstash_system (for cluster health) + logstash_admin (for .logstash ) + ?? role (for .monitoring-logstash ) |
Human | logastash_user | logstash_reader (for logstash ) |
Human | logstash_user_admin | logstash_reader (for logstash ) + logstash_admin (for .logstash ) + monitoring_user (for .monitoring-logstash ) |
bin/x-pack/setup-passwords interactive
^^ is needed to bootstrap the logstash_system
user.
logstash_system The user Logstash uses when storing monitoring information in Elasticsearch.
so I think that bootstrapping step is needed to give the .monitoring-logstash*
write privs ... but it is unclear how to do that for cloud deployment.
?? role (for .monitoring-logstash)
I confirmed that the logstash_system
role has implicit privs to write to the monitoring index.
Per the proposal above I suggest we should (by default) give the logstash_system
user the logstash_admin
role to as well to allow (by default) the logstash_system
user to be used for both monitoring and management.
is needed to bootstrap the logstash_system user.
Also confirmed that this user is not currently available via cloud, so we will likely want to explicitly document how to set this user up. (note the role is present , just not the user)
For
.logstash*
, there are two personas:the Logstash system reading the management data the Human changing the central config (management data)
I agree with the personas.
(note we don't to support reading the central config as standalone persona due to security reasons)
Can you elaborate on this? I've always wondered about this: why the Logstash system doesn't get just read access to .logstash
, because it doesn't need to write to it. From a security POV, I'd think we'd want to provide as restrictive access as necessary, no?
Can you elaborate on this? I've always wondered about this: why the Logstash system doesn't get just read access to .logstash, because it doesn't need to write to it
The comment was in reference to the human use case, where we should never give a human the ability to read but not write. Doing that would open the door to accidentally give a non-admin user the ability see potentially sensitive information.
I wasn't part of the part the decision w/r/t the definition of the role, but it does make sense for the human use case.
From a security POV, I'd think we'd want to provide as restrictive access as necessary, no?
Agreed for the system use case. Perhaps we should try address that with a change to the logstash_system
role to provide that role with the read access .logstash
? _(that would mean in a minor change to the proposed logstash_systemoutput user above)
The comment was in reference to the human use case, where we should never give a human the ability to read but not write. Doing that would open the door to accidentally give a non-admin user the ability see potentially sensitive information.
The comment was in reference to the human use case, where we should never give a human the ability to read but not write. Doing that would open the door to accidentally give a non-admin user the ability see potentially sensitive information.
I wasn't part of the part the decision w/r/t the definition of the role, but it does make sense for the human use case.
Agreed. Makes sense. Thanks for the explanation.
Perhaps we should try address that with a change to the
logstash_system
role to provide that role with the read access.logstash
? _(that would mean in a minor change to the proposed logstash_systemoutput user above)
Yeah, this makes sense to me. It does mean that all existing users of Logstash would've given their Logstash (system) read and write privileges to .logstash
but at least new users would have the option of using a more restricted user (logstash_system_output
).
For the Beats -> LS -> ES (with secured cluster) use case, we also need to document the privileges that are required to write/read to beats indexes for users who want to use the beats dashboards--that is, users who have this config:
output {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
user => "logstash_internal"
password => "x-pack-test-password"
}
}
You get the following security exception if the user doesn't have the correct privileges:
[2018-05-17T18:35:26,780][INFO ][logstash.outputs.elasticsearch] retrying failed action with
response code: 403 ({"type"=>"security_exception", "reason"=>"action [indices:admin/create] is
unauthorized for user [logstash_internal]"})
Hello, I am exactly running into the above issue using stack 6.3.1.
I have a secured cluster with almost similar config.
Is this issue resolved now?
][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"security_exception", "reason"=>"action [indices:admin/create] is unauthorized for user [logstash_internal]"})
@ashokbellur Make sure your logasth_internal
user has write
and create_index
privileges on the Beats index you are trying to send events to. Note that this GitHub issue is for tracking updates required to the documentation. If you have questions about configuring Logstash and/or Beats, please ask in the discussion forums. See:
https://discuss.elastic.co/c/logstash https://discuss.elastic.co/c/beats
@karenzone Is this issue still valid, or should I close it?
closing because this issue is soooo old.
Currently we document how to set up users for Logstash (here), but we don't document which roles are required when using specific features, like the pipeline viewer and monitoring.
(One exception is centralized config management, where we do document the required roles: https://www.elastic.co/guide/en/logstash/master/logstash-centralized-pipeline-management.html)
@karenzone You'll need to investigate which roles are required for using the pipeline viewer and monitoring.