lukewaite
commented
7 years ago I think you're on the right track. The config for the aws sdk is handled entirely by the logstash-mixin-aws gem.
Are you using any other plugins which interact with the AWS SDK in some way? If so, are they working? Which are they?
It looks like you can set an http_proxy option on the mixin (just apply this to your config of this plugin). Rather than setting the proxy and exception as java options, can you try configuring the plugin to use the proxy with this setting?
https://github.com/logstash-plugins/logstash-mixin-aws/blob/master/lib/logstash/plugin_mixins/aws_config/v2.rb#L20
I' am getting the "Error: unable to sign request without credentials set" error if I do not supply an AWS access key id/secret access key (even though the instance is in a role). I have been researching the issue and have a theory but not the time to verify unfortunately. I also understand the issue may live upstream with the logstash-mixin-aws gem.
My servers are in a VPC with an HTTP proxy (no direct access to the internet). I have tried supplying proxy information via LS_JAVA_OPTIONS env var and /etc/logstash/jvm.options.
I also set -DnonProxyHost=169.254.169.254 so that the service should NOT use the proxy for Ec2 instance metadata queries. Anything that needs the role credentials would query http://169.254.169.254/latest/meta-data/iam/security-credentials/$ROLENAME to get the access key id and secretaccesskey which leads me to my theory: Either this plugin or the logstash-mixin-aws gem is not respecting the nonProxyHost jvm option.
Let me know if this sounds sane. Thanks for your time
I start logstash with this: sudo -Hu logstash bin/logstash -f /etc/logstash/conf.d/logstash.conf --path.settings /etc/logstash