Environment (please complete the following information):
AWS-managed OpenSearch: v2.9.0
Data Prepper: v2.6.2
AWS EKS: v1.29
Additional context
When I map the same user to all_access role, Data Prepper behaves as expected, and is able to set up the pipeline, listens on the configured port, and successfully writes to OpenSearch. If I add index permissions of indices_all on pattern *, it still doesn't work. So it's maybe cluster level index permissions?
I've set JAVA_OPTS=-Dlog4j2.debug=true so hopefully the logs show something useful, but not the failing API call that I can see.
Describe the bug
When configuring an OpenSearch sink with a user, the permissions documented result in failure to write data to OpenSearch
To Reproduce
Configure pipeline sink:
Configure role:
Map role to user:
Expected behavior
Data prepper starts without error, listens on the appropriate port, and writes to OpenSearch
Screenshots
As above.
log gist
Environment (please complete the following information):
Additional context
When I map the same user to
all_access
role, Data Prepper behaves as expected, and is able to set up the pipeline, listens on the configured port, and successfully writes to OpenSearch. If I add index permissions ofindices_all
on pattern*
, it still doesn't work. So it's maybe cluster level index permissions? I've setJAVA_OPTS=-Dlog4j2.debug=true
so hopefully the logs show something useful, but not the failing API call that I can see.