Open bogdandisc opened 1 year ago
Hi @bogdandisc have You resolved the issue?
Unfortunately I have similar issue and according to the AWS documentation You have to attach proper policy to the OS, and additionally You have to enable AWS Signature Version 4.
\
Even if you configure a completely open resource-based access policy, all requests to the OpenSearch Service configuration API must be signed. If your policies specify IAM roles or users, requests to the OpenSearch APIs also must be signed using AWS Signature Version 4.
I found in Fluentbit Docuemtnation the parameter AWS_Auth - Enable AWS Sigv4 Authentication for Amazon OpenSearch Service
but I use helm chart and to be honest I don't know how to use this parameter in my case... Below section from values.yaml
clusterOutputs:
- name: opensearch-output
spec:
opensearch:
host: vpc-domain-name-xxxxxxxxxxxxx.us-east-1.es.amazonaws.com
port: 443
scheme: https
logstash_format: true
logstash_prefix: ${tag}
index_name: ${tag}-%Y.%m.%d
include_timestamp: true
reconnect_on_error: true
log_os_400_reason: true
buffer:
path: /buffers/opensearch
type: file
timekey: 1m
timekey_wait: 30s
timekey_use_utc: true
Do You have any suggestions/ideas?
Thanks
have you given necessary wite grants on opensearch side? you have to map your role with necessary permission https://opensearch.org/docs/latest/security/access-control/users-roles/ Creation of k8s secrets was not necessary from my experience In your output conf you have also to add AWS_Auth On AWS_Region eu-west-1
Environment:
I am trying to deploy fluent-bit using the standard helm chart:
Have tried to create a new service account via the helm chart:
And also tried to create a new service account and attach it in the values.yaml file. In both cases I can see the service account getting attached to the fluent-bit pods and the trust policy working. There's no reason that the IRSA token is not used by the fluent-bit role.
I am also aware that in 1.24 secrets are not created automatically, so I am creating one in terraform and attaching it to the service account:
However the fluent-bit pods are not authenticating using the
fluent-bit
service account and are using the worker node IAM role instead: