Trouble using vagrant-aws with my account - permissions - what fails?

[lollisoft]

Hi,

I tried hard to figure out why my setup of a vagrant aws box failed to provision. Generally it tells me the following error: UnauthorizedOperation => You are not authorized to perform this operation.

Checking the credentials I have created by the simulation tool, I believe that these are correct. Passing wrong credentials also approves, that I am correctly logging in when the correct ones are passed.

My main problem is that I do not see any verbose logging, so I am unable to see when or by what command failures happens. Here is my output (using a private AMI):

stsmac:~ lothar$ export VAGRANT_LOG=debug & sudo vagrant up --provider=aws --no-parallel [1] 45196 Password: Bringing machine 'default' up with 'aws' provider... WARNING: Nokogiri was built against LibXML version 2.8.0, but has dynamically loaded 2.9.1 [fog][WARNING] Unable to load the 'unf' gem. Your AWS strings may not be properly encoded. [default] Warning! The AWS provider doesn't support any of the Vagrant high-level network configurations (config.vm.network). They will be silently ignored. [default] Launching an instance with the following settings... [default] -- Type: m1.small [default] -- AMI: ami-92d524e5 [default] -- Region: us-east-1 [default] -- Keypair: [default] -- Block Device Mapping: [] [default] -- Terminate On Shutdown: false [default] -- Monitoring: false [default] -- EBS optimized: false There was an error talking to AWS. The error message is shown below:

UnauthorizedOperation => You are not authorized to perform this operation. [1]+ Done export VAGRANT_LOG=debug stsmac:~ lothar$

How can I see the issued comands?

Thanks, Lothar

[hlascelles]

Same problem here... I've added the permissions you might expect (StartInstances,RunInstances,StopInstances,TerminateInstances,DescribeInstances,DescribeInstanceStatus,RebootInstances) and still no joy. It's hard to tell what it is choking on.

[davidjb]

I'm curious on this front as well. I've tried the same as @hlascelles but no success. I've ended up taking to enabling the 'Power user' policy and it works for now.

[amiorin]

Same problem here.

[rjocoleman]

I've come across this today too. It's quite specific to the IAM policy, to diagnose what permission you're missing rather than trial and error you can enable AWS's CloudTrail

Once enabled run a vagrant up --provider aws, it should fail as normal. Inspect your CloudTrail JSON log file, it can take up to 15 minutes for CloudTrail logs to appear. CloudTrail can be very noisy but you narrow searches by access key or similar. This will show what API calls your making and what response (if errors) are being given. You may need to repeat this several times as new calls fail.

In my case I was missing DescribeSecurityGroups.

[msgilligan]

This has been the single most frustrating issue with using Vagrant on AWS. If there's any way to provide information as to which operation was unauthorized, that would be very helpful.

CloudTrail helps, but wouldn't be necessary if the direct error message were more informatiive.

[rhbecker]

I agree that a more verbose failure message and/or logging would be useful.

In case it helps someone else, I setup a policy per that suggested by the packer docs, adding only DescribeSecurityGroups, as mentioned by @rjocoleman. That worked for me.

[PaulTurner-awin]

I am seeing the same error message and I have permissions to "ec2:*". Any ideas?

[hlascelles]

We gave up ultimately and now use packer with bash, which works great. Vagrant and Chef have left the building (where we work).

[rjocoleman]

@PaulTurner-awin as I said in https://github.com/mitchellh/vagrant-aws/issues/199#issuecomment-45586798 you can see all the calls via CloudTrail if you need to.

Sorry I'm unsure if more verbose logging has come in to core vagrant-aws as I haven't upgraded in some time!

That said, I have a vague recollection you might also need iam:PassRole

[PaulTurner-awin]

I found this article really useful; http://blogs.aws.amazon.com/security/post/Tx2KPWZJJ4S26H6/Demystifying-EC2-Resource-Level-Permissions in sorting out the permissions. I still have some issues with vagrant-aws not working, but I now have the kitchen-ec2 driver working in test kitchen (http://kitchen.ci) so can create and destroy instances for testing Chef cookbooks. Will look at resolving vagrant-aws issues another time as it is not so urgent for me now.

[scottalan]

For me it seems I have to use:

security_groups:
                    - default

which of course leaves it wide open.

From here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html#vpc-security-groups

*When you specify a security group for a nondefault VPC to the CLI 
or the API actions, you must use the security group ID and not the 
security group name to identify the security group.

Tried that and it still gives the error

[scottalan]

I get past the initial error but it hangs when it is trying to connect via SSH:

DEBUG ssh: == Net-SSH connection debug-level log END ==
 INFO retryable: Retryable exception raised: #<Timeout::Error: execution expired>
 INFO ssh: Attempting to connect to SSH...
 INFO ssh:   - Host: 11.111.11.111
 INFO ssh:   - Port: 22
 INFO ssh:   - Username: ubuntu
 INFO ssh:   - Password? false
 INFO ssh:   - Key Path: ["/Users/user/.ssh/ec2.pem"]

It just tries over and over and over. I can see the instance is created.

[twolfson]

After a lot of trial/error (we wound up doing a binary search with ec2:A* ... ec2:Z* for policies), we finally got our IAM policy right. We used the Demystifying article as well.

Here's our Vagrant setup:

# Allow AWS provisioning (used by Jenkins)
config.vm.provider "aws" do |aws, override|
  # Use an empty box that says "provider": "aws"
  # https://github.com/mitchellh/vagrant-aws/tree/v0.6.0#quick-start
  config.vm.box = "https://github.com/mitchellh/vagrant-aws/raw/v0.6.0/dummy.box"

  # Declare our credentials
  aws.access_key_id = "{{access key id}}"
  aws.secret_access_key = "{{secret access key}}"

  # Use our AWS user and SSH key
  aws.keypair_name = "{{keypair name}}"
  override.ssh.username = "{{username}}"
  override.ssh.private_key_path = "{{path-to-ssh-key}}"

  # Use security group with SSH access
  # DEV: We require a private IP address for access to work with security groups
  aws.security_groups = ["{{security-group-name}}"]
  aws.ssh_host_attribute = :private_ip_address

  # Specify the type of server we want
  aws.ami = "{{image}}"
  aws.instance_type = "{{instance-size}}"
  aws.region = "{{region}}"

  # Tag the server for easy searching
  aws.tags = {
    :type => '{{tag}}'
  }
end

Here's the corresponding IAM policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "TheseActionsDontSupportResourceLevelPermissions",
            "Effect": "Allow",
            "Action": [
                "ec2:CreateTags",
                "ec2:AssignPrivateIpAddresses",
                "ec2:Describe*",
                "ec2:UnassignPrivateIpAddresses"
            ],
            "Resource": "*"
        },
        {
            "Sid": "ThisActionsSupportResourceLevelPermissions",
            "Effect": "Allow",
            "Action": [
                "ec2:RunInstances"
            ],
            "Resource": [
                "arn:aws:ec2:{{region}}:{{account id without dashes}}:instance/*",
                "arn:aws:ec2:{{region}}:{{account id without dashes}}:key-pair/*",
                "arn:aws:ec2:{{region}}:{{account id without dashes}}:network-interface/*",
                "arn:aws:ec2:{{region}}:{{account id without dashes}}:security-group/*",
                "arn:aws:ec2:{{region}}:{{account id without dashes}}:subnet/*",
                "arn:aws:ec2:{{region}}:{{account id without dashes}}:volume/*",
                "arn:aws:ec2:{{region}}::image/ami-*"
            ]
        },
        {
            "Sid": "TheseActionsSupportResourceLevelPermissions",
            "Effect": "Allow",
            "Action": [
                "ec2:TerminateInstances",
                "ec2:StopInstances",
                "ec2:StartInstances"
            ],
            "Resource": "arn:aws:ec2:{{region}}:{{account id without dashes}}:instance/*"
        }
    ]
}

Notes from this, Amazon does provide error messages for failing RunInstances. These can be decoded via aws sts as documented in the Demystifying article. However, there's also a useless error message "UnauthorizedOperation => You are not authorized to perform this operation."

It turns out that we needed ec2:CreateTags for that. It's likely due to us using aws.tags in the Vagrantfile.

[jFrenetic]

@twolfson thanks a lot, this helped me to solve my problems