Can I get better documentation/Absolutly nothing works

[rlewkowicz]

I'm hoping this doesn't get closed as "go ask somewhere else" because I cannot get past a simple 403.

I'm not a stupid person, have followed the most basic directions and just nothing works.

[2020-06-07T15:20:38,961][WARN ][logstash.outputs.amazonelasticsearch][main] Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"https://vpc-rauserstest-sputmajchrtx7yv3wlluh7l5s4.us-east-1.es.amazonaws.com:443/", :error_type=>LogStash::Outputs::AmazonElasticSearch::HttpClient::Pool::BadResponseCodeError, :error=>"Got response code '403' contacting Elasticsearch at URL 'https://vpc-rauserstest-sputmajchrtx7yv3wlluh7l5s4.us-east-1.es.amazonaws.com:443/'"}

I've tried with and without basic auth. My policy cant really get any more permissive.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:us-east-1:206553244089:domain/rauserstest/*"
    }
  ]
}

My user is an admin. I literally cannot make my user any more powerful or my permissions any more open. I created client creds for this user. My plugin stanza (Which by the way, absolutely absurd I have to use a plugin):

output {
    stdout { codec =>  "rubydebug"}
    amazon_es {
        hosts => "vpc-rauserstest-sputmajchrtx7yv3wlluh7l5s4.us-east-1.es.amazonaws.com"
        ssl => "true"
        region => "us-east-1"
        # user => 'redact'
        # password => 'redact'
        # aws_access_key_id and aws_secret_access_key are optional if instance profile is configured
        aws_access_key_id => ' redact'
        aws_secret_access_key => 'redact'
        index => "test"
    }
} 

[AustinTag]

I notice your cluster is in a VPC. Is the instance on which youre running logstash in that same VPC?

[AustinTag]

Also, since you've opened the cluster to not need AWS credentials for auth, can you hit the ES cluster with a curl command from that instance that you're running logstash on?

[rlewkowicz]

I use "sshuttle" to create a transparent proxy to a jump box so I can do development from my laptop. It did cross my mind that somehow the iam user of that box might be coming into play, so I did add es:* to that user (which is a different user than my client credentials are for). The readme states that the first credential verified is:

User passed aws_access_key_id and aws_secret_access_key in amazon_es configuration

My preference would be to not use any iam profiles and have pure basic auth as the only thing protecting the instance. It would have been nice to do a 60 minute POC. Turn a thing on, connect to it, done. Tear it down.

For now, I think I'm just going to go pull the bitnami docker images and stand up a basic node myself.

[AustinTag]

Just in case you do want to play with Amazon Elasticsearch again:

The way that you had set up your access policy (the fact that it was open to any AWS principal) means that IAM credentials were not needed, nor did they come into play. I believe your issue likely had to do with your cluster being in a VPC and trying access the cluster from outside of the VPC. Feel free to re-open the issue if you have any other questions.