search

k1LoW / awsecrets

AWS credentials loader
MIT License
17 stars 9 forks source link

LoadSecrets=true seems to be not taken into account when used from CodeBuild #20

Open nckbnv opened 6 years ago

nckbnv commented 6 years ago

Hi there,

I've stumbled upon a strange behaviour of awspec/awsecrets depending on the environment from which you are executing the awspec tests.

Test 1 - Using an EC2 instance with IAM role and secrets load disabled:

require 'spec_helper'

describe ec2('docker-apache-poc') do
  it { should be_running }
  its(:image_id) { should eq 'ami-0bdb1d6c15a40392c' }
  it { should have_security_group('docker-apache-sg') }
end
[root@ip-172-31-3-183 spec]# cat spec_helper.rb 
require 'awspec'
require 'awsecrets'

Awsecrets.load(disable_load_secrets:true)
#Awsecrets.load(secrets_path: File.expand_path('./secrets.yml', File.dirname(__FILE__)))

This works fine:

[root@ip-172-31-3-183 spec]# bundle exec rake spec
(in /home/ec2-user/test2)
/usr/local/rvm/rubies/ruby-2.5.1/bin/ruby -I/usr/local/rvm/gems/ruby-2.5.1/gems/rspec-core-3.8.0/lib:/usr/local/rvm/gems/ruby-2.5.1/gems/rspec-support-3.8.0/lib /usr/local/rvm/gems/ruby-2.5.1/gems/rspec-core-3.8.0/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb

ec2 'docker-apache-inspec-poc'
  should be running
  should have security group "docker-apache-sg"
  image_id
    should eq "ami-0bdb1d6c15a40392c"

Finished in 0.22159 seconds (files took 1.56 seconds to load)
3 examples, 0 failures

Same code when started from AWS CodeBuild (with the same policies applied) however fails with the following output:

    Aws::Errors::MissingCredentialsError:
       unable to sign request without credentials set
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/signature_v4.rb:119:in `rescue in apply_signature'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/signature_v4.rb:111:in `apply_signature'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/signature_v4.rb:65:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/helpful_socket_errors.rb:10:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/retry_errors.rb:138:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/query/handler.rb:28:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/user_agent.rb:13:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/seahorse/client/plugins/endpoint.rb:45:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/param_validator.rb:24:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/seahorse/client/plugins/raise_response_errors.rb:14:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/idempotency_token.rb:17:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/param_converter.rb:24:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/response_paging.rb:10:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/seahorse/client/plugins/response_target.rb:23:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/seahorse/client/request.rb:70:in `send_request'
     # /usr/local/bundle/gems/aws-sdk-ec2-1.50.0/lib/aws-sdk-ec2/client.rb:11759:in `describe_instances'
     # /usr/local/bundle/gems/awspec-1.9.0/lib/awspec/helper/client_wrap.rb:26:in `method_missing'
     # /usr/local/bundle/gems/awspec-1.9.0/lib/awspec/helper/finder/ec2.rb:13:in `rescue in find_ec2'
     # /usr/local/bundle/gems/awspec-1.9.0/lib/awspec/helper/finder/ec2.rb:6:in `find_ec2'
     # /usr/local/bundle/gems/awspec-1.9.0/lib/awspec/type/ec2.rb:12:in `resource_via_client'
     # /usr/local/bundle/gems/awspec-1.9.0/lib/awspec/type/ec2.rb:30:in `block (2 levels) in <class:Ec2>'
     # ./spec/ec2_spec.rb:4:in `block (2 levels) in <top (required)>'
     # ------------------
     # --- Caused by: ---
     # Aws::Sigv4::Errors::MissingCredentialsError:
     #   unable to sign request without credentials set
     #   /usr/local/bundle/gems/aws-sigv4-1.0.3/lib/aws-sigv4/signer.rb:570:in `get_credentials'

  2) ec2 'docker-apache-poc' should have security group "docker-apache-sg"
     Failure/Error: it { should have_security_group('docker-apache-sg') }

     Aws::Errors::MissingCredentialsError:
       unable to sign request without credentials set
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/signature_v4.rb:119:in `rescue in apply_signature'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/signature_v4.rb:111:in `apply_signature'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/signature_v4.rb:65:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/helpful_socket_errors.rb:10:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/retry_errors.rb:138:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/query/handler.rb:28:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/user_agent.rb:13:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/seahorse/client/plugins/endpoint.rb:45:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/param_validator.rb:24:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/seahorse/client/plugins/raise_response_errors.rb:14:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/idempotency_token.rb:17:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/param_converter.rb:24:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/response_paging.rb:10:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/seahorse/client/plugins/response_target.rb:23:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/seahorse/client/request.rb:70:in `send_request'
     # /usr/local/bundle/gems/aws-sdk-ec2-1.50.0/lib/aws-sdk-ec2/client.rb:11759:in `describe_instances'
     # /usr/local/bundle/gems/awspec-1.9.0/lib/awspec/helper/client_wrap.rb:26:in `method_missing'
     # /usr/local/bundle/gems/awspec-1.9.0/lib/awspec/helper/finder/ec2.rb:13:in `rescue in find_ec2'
     # /usr/local/bundle/gems/awspec-1.9.0/lib/awspec/helper/finder/ec2.rb:6:in `find_ec2'
     # /usr/local/bundle/gems/awspec-1.9.0/lib/awspec/type/ec2.rb:12:in `resource_via_client'
     # /usr/local/bundle/gems/awspec-1.9.0/lib/awspec/type/ec2.rb:59:in `has_security_group?'
     # ./spec/ec2_spec.rb:6:in `block (2 levels) in <top (required)>'
     # ------------------
     # --- Caused by: ---
     # Aws::Sigv4::Errors::MissingCredentialsError:
     #   unable to sign request without credentials set
     #   /usr/local/bundle/gems/aws-sigv4-1.0.3/lib/aws-sigv4/signer.rb:570:in `get_credentials'

  3) ec2 'docker-apache-inspec-poc' image_id 
     Failure/Error: its(:image_id) { should eq 'ami-0bdb1d6c15a40392c' }

     Aws::Errors::MissingCredentialsError:
       unable to sign request without credentials set
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/signature_v4.rb:119:in `rescue in apply_signature'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/signature_v4.rb:111:in `apply_signature'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/signature_v4.rb:65:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/helpful_socket_errors.rb:10:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/retry_errors.rb:138:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/query/handler.rb:28:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/user_agent.rb:13:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/seahorse/client/plugins/endpoint.rb:45:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/param_validator.rb:24:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/seahorse/client/plugins/raise_response_errors.rb:14:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/idempotency_token.rb:17:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/param_converter.rb:24:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/aws-sdk-core/plugins/response_paging.rb:10:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/seahorse/client/plugins/response_target.rb:23:in `call'
     # /usr/local/bundle/gems/aws-sdk-core-3.30.0/lib/seahorse/client/request.rb:70:in `send_request'
     # /usr/local/bundle/gems/aws-sdk-ec2-1.50.0/lib/aws-sdk-ec2/client.rb:11759:in `describe_instances'
     # /usr/local/bundle/gems/awspec-1.9.0/lib/awspec/helper/client_wrap.rb:26:in `method_missing'
     # /usr/local/bundle/gems/awspec-1.9.0/lib/awspec/helper/finder/ec2.rb:13:in `rescue in find_ec2'
     # /usr/local/bundle/gems/awspec-1.9.0/lib/awspec/helper/finder/ec2.rb:6:in `find_ec2'
     # /usr/local/bundle/gems/awspec-1.9.0/lib/awspec/type/ec2.rb:12:in `resource_via_client'
     # /usr/local/bundle/gems/awspec-1.9.0/lib/awspec/type/base.rb:39:in `method_missing'
     # ./spec/ec2_spec.rb:5:in `block (2 levels) in <top (required)>'
     # ------------------
     # --- Caused by: ---
     # Aws::Sigv4::Errors::MissingCredentialsError:
     #   unable to sign request without credentials set
     #   /usr/local/bundle/gems/aws-sigv4-1.0.3/lib/aws-sigv4/signer.rb:570:in `get_credentials'

Finished in 0.01336 seconds (files took 1.41 seconds to load)
3 examples, 3 failures

Failed examples:

rspec ./spec/ec2_spec.rb:4 # ec2 'docker-apache-poc' should be running
rspec ./spec/ec2_spec.rb:6 # ec2 'docker-apache-poc' should have security group "docker-apache-sg"
rspec ./spec/ec2_spec.rb:5 # ec2 'docker-apache-poc' image_id 

/usr/local/bin/ruby -I/usr/local/bundle/gems/rspec-core-3.8.0/lib:/usr/local/bundle/gems/rspec-support-3.8.0/lib /usr/local/bundle/gems/rspec-core-3.8.0/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb failed

[Container] 2018/10/05 13:23:22 Command did not exit successfully bundle exec rake spec exit status 1

I was able to successfully verify that the container is using the correct role.

And I was able to successfully run some inspec tests from the same container. And as inspec is another ruby application I would assume that they are using aws-sdk in the backend too.

Any ideas of what the issue is?

I will be glad to provide you with some more info if required.