opensearch-project / logstash-output-opensearch

A Logstash plugin that sends event data to a OpenSearch clusters and stores as an index.
https://opensearch.org/docs/latest/clients/logstash/index/
Apache License 2.0
106 stars 80 forks source link

[BUG] Cross account iam roles do not seem to work #263

Open pastibog opened 5 days ago

pastibog commented 5 days ago

Describe the bug Trying to send data from a EC2 instance to a OS cluster in a different account with IAM roles don't seem to work.

To Reproduce

Account B opensearch_role:

{
    "Statement": [
        {
            "Action": "es:*",
            "Effect": "Allow",
            "Resource": [
                "domain_arn"
            ]
        }
    ],
    "Version": "2012-10-17"
}

Account B opensearch role trust relationship:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<account_a>:root"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Redacted logstash config:

  opensearch {
    hosts => ["https://host:443"]
    index  => "test-name-%{+YYYY.MM.dd}"
    ecs_compatibility => disabled
    auth_type => {
      type => "aws_iam"
      region => "eu-west-1"
      service_name => "es"
      profile => "research"
    }
  }
}

We do something similar with fluent-bit where that works fine. Was wondering if this is not supported right now.

pastibog commented 2 days ago

Forgot to mention that our config file looks like this:

[default]
region = <region>
credential_source = Ec2InstanceMetadata

[profile research]
source_profile = default
region = <region>
role_arn = arn:aws:iam::XXXXXXXXXXXX:role/role_in_account_B