Feature Request: support attaching EBS volumes

justinclayton commented 11 years ago

Proof of concept as to how this might look can be seen in my fork here:

https://github.com/justinclayton/vagrant-aws/commit/ee062208fade734ac832c76f043bac081706d64d

briandoconnor commented 11 years ago

Also, I'm not sure if someone has mentioned this before but there seems to be no way to specify ephemeral disks and how they should be exposed to the instance. Need something in the config similar to what's documented here:

http://serverfault.com/questions/335727/missing-ephemeral-disks-on-an-m1-xlarge-instance

Right now it looks like I can fire up the instance type I want but I can't access the 1TB+ of local instance disks, they simply aren't exposed to the Linux OS unless the instance is launched with the ephemerial[0-4] configured properly. My understanding is this can't be done after the fact (like EBS volumes) and has to be done at launch time.

MatthaeusHarris commented 11 years ago

Ephemeral volume support is in https://github.com/mitchellh/vagrant-aws/pull/93

n0ts commented 11 years ago

EC2 RunInstances API support BlockDeviceMapping.n.Ebs.*. http://docs.aws.amazon.com/AWSEC2/latest/APIReference/ApiReference-query-RunInstances.html

I tried bellow Vagrantfile, "The request received was invalid.".

aws.block_device_mapping = [
  {
    :DeviceName => "/dev/sdf",
    :Ebs => {
      "DeleteOnTermination"=> false,
      "VolumeSize" => "100",
      "VolumeType" => "Standard",
    }
]
...

tralamazza commented 11 years ago

@n0ts which vagrant-aws version are you using? Could you please paste the entire log (running with VAGRANT_LOG=debug)? Thanks.

n0ts commented 11 years ago

vagrant version is 1.2.7.

Here is the vagrant debug log.

 INFO global: Vagrant version: 1.2.7
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/commands/box/plugin.rb
 INFO manager: Registered plugin: box command
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/commands/destroy/plugin.rb
 INFO manager: Registered plugin: destroy command
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/commands/halt/plugin.rb
 INFO manager: Registered plugin: halt command
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/commands/help/plugin.rb
 INFO manager: Registered plugin: help command
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/commands/init/plugin.rb
 INFO manager: Registered plugin: init command
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/commands/package/plugin.rb
 INFO manager: Registered plugin: package command
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/commands/plugin/plugin.rb
 INFO manager: Registered plugin: plugin command
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/commands/provision/plugin.rb
 INFO manager: Registered plugin: provision command
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/commands/reload/plugin.rb
 INFO manager: Registered plugin: reload command
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/commands/resume/plugin.rb
 INFO manager: Registered plugin: resume command
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/commands/ssh/plugin.rb
 INFO manager: Registered plugin: ssh command
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/commands/ssh_config/plugin.rb
 INFO manager: Registered plugin: ssh-config command
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/commands/status/plugin.rb
 INFO manager: Registered plugin: status command
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/commands/suspend/plugin.rb
 INFO manager: Registered plugin: suspend command
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/commands/up/plugin.rb
 INFO manager: Registered plugin: up command
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/communicators/ssh/plugin.rb
 INFO manager: Registered plugin: ssh communicator
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/guests/arch/plugin.rb
 INFO manager: Registered plugin: Arch guest
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/guests/debian/plugin.rb
 INFO manager: Registered plugin: Debian guest
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/guests/fedora/plugin.rb
 INFO manager: Registered plugin: Fedora guest
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/guests/freebsd/plugin.rb
 INFO manager: Registered plugin: FreeBSD guest
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/guests/gentoo/plugin.rb
 INFO manager: Registered plugin: Gentoo guest
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/guests/linux/plugin.rb
 INFO manager: Registered plugin: Linux guest.
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/guests/omnios/plugin.rb
 INFO manager: Registered plugin: OmniOS guest.
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/guests/openbsd/plugin.rb
 INFO manager: Registered plugin: OpenBSD guest
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/guests/pld/plugin.rb
 INFO manager: Registered plugin: PLD Linux guest
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/guests/redhat/plugin.rb
 INFO manager: Registered plugin: RedHat guest
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/guests/solaris/plugin.rb
 INFO manager: Registered plugin: Solaris guest.
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/guests/suse/plugin.rb
 INFO manager: Registered plugin: SUSE guest
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/guests/ubuntu/plugin.rb
 INFO manager: Registered plugin: Ubuntu guest
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/hosts/arch/plugin.rb
 INFO manager: Registered plugin: Arch host
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/hosts/bsd/plugin.rb
 INFO manager: Registered plugin: BSD host
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/hosts/fedora/plugin.rb
 INFO manager: Registered plugin: Fedora host
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/hosts/freebsd/plugin.rb
 INFO manager: Registered plugin: FreeBSD host
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/hosts/gentoo/plugin.rb
 INFO manager: Registered plugin: Gentoo host
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/hosts/linux/plugin.rb
 INFO manager: Registered plugin: Linux host
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/hosts/opensuse/plugin.rb
 INFO manager: Registered plugin: OpenSUSE host
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/hosts/windows/plugin.rb
 INFO manager: Registered plugin: Windows host
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/kernel_v1/plugin.rb
 INFO manager: Registered plugin: kernel
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/kernel_v2/plugin.rb
 INFO manager: Registered plugin: kernel
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/providers/virtualbox/plugin.rb
 INFO manager: Registered plugin: VirtualBox provider
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/provisioners/ansible/plugin.rb
 INFO manager: Registered plugin: ansible
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/provisioners/cfengine/plugin.rb
 INFO manager: Registered plugin: CFEngine Provisioner
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/provisioners/chef/plugin.rb
 INFO manager: Registered plugin: chef
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/provisioners/puppet/plugin.rb
 INFO manager: Registered plugin: puppet
DEBUG global: Loading core plugin: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/provisioners/shell/plugin.rb
 INFO manager: Registered plugin: shell
 INFO vagrant: `vagrant` invoked: ["up", "--provider=aws"]
DEBUG vagrant: Creating Vagrant environment
 INFO environment: Environment initialized (#)
 INFO environment:   - cwd: //aws_test
 INFO environment: Home path: /Users/n0ts/.vagrant.d
 INFO environment: Local data path: //aws_test/.vagrant
DEBUG environment: Creating: //aws_test/.vagrant
DEBUG environment: Loading plugins from: /Users/n0ts/.vagrant.d/plugins.json
 INFO environment: Loading plugin from JSON: sahara
 INFO manager: Registered plugin: sahara
 INFO environment: Loading plugin from JSON: vagrant-aws
 INFO manager: Registered plugin: AWS
 INFO environment: Loading plugin from JSON: vagrant-berkshelf
 INFO manager: Registered plugin: berkshelf
 INFO environment: Loading plugin from JSON: vagrant-digitalocean
 INFO manager: Registered plugin: DigitalOcean
 INFO environment: Loading plugin from JSON: vagrant-lxc
 INFO manager: Registered plugin: Linux Containers (LXC) provider
 INFO environment: Loading plugin from JSON: vagrant-managed-servers
 INFO manager: Registered plugin: ManagedServers
 INFO environment: Loading plugin from JSON: vagrant-ohai
 INFO manager: Registered plugin: vagrant-ohai
 INFO environment: Loading plugin from JSON: vagrant-omnibus
 INFO manager: Registered plugin: Omnibus
 INFO environment: Loading plugin from JSON: vagrant-rackspace
 INFO manager: Registered plugin: RackSpace Cloud
 INFO environment: Running hook: environment_load
 INFO environment: Initializing config...
 INFO loader: Set :default = "/Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/config/default.rb"
DEBUG loader: Populating proc cache for "/Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/config/default.rb"
DEBUG loader: Load procs for pathname: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/config/default.rb
 INFO loader: Set :root = #
DEBUG loader: Populating proc cache for #
DEBUG loader: Load procs for pathname: //aws_test/Vagrantfile
 INFO loader: Loading configuration in order: [:default, :home, :root]
DEBUG loader: Loading from: default (evaluating)
DEBUG loader: Loading from: root (evaluating)
DEBUG loader: Configuration loaded successfully, finalizing and returning
DEBUG hosts: Host path search classes: [VagrantPlugins::HostWindows::Host, VagrantPlugins::HostOpenSUSE::Host, VagrantPlugins::HostArch::Host, VagrantPlugins::HostFedora::Host, VagrantPlugins::HostFreeBSD::Host, VagrantPlugins::HostGentoo::Host, VagrantPlugins::HostLinux::Host, VagrantPlugins::HostBSD::Host]
 INFO hosts: Host class: VagrantPlugins::HostBSD::Host
 INFO runner: Preparing hooks for middleware sequence...
 INFO runner: 1 hooks defined.
 INFO runner: Running action: #
 INFO cli: CLI: [] "up" ["--provider=aws"]
DEBUG cli: Invoking command class: VagrantPlugins::CommandUp::Command ["--provider=aws"]
DEBUG command: 'Up' each target VM...
DEBUG command: Getting target VMs for command. Arguments:
DEBUG command:  -- names: []
DEBUG command:  -- options: {:provider=>"aws"}
DEBUG command: Loading all machines...
 INFO environment: Getting machine: test1 (aws)
 INFO environment: Uncached load of machine.
 INFO loader: Set :vm_test1 = [["2", #]]
DEBUG loader: Populating proc cache for ["2", #]
 INFO loader: Loading configuration in order: [:default, :home, :root, :vm_test1]
DEBUG loader: Loading from: default (cache)
DEBUG loader: Loading from: root (cache)
DEBUG loader: Loading from: vm_test1 (evaluating)
DEBUG provisioner: Provisioner defined: chef_solo
DEBUG loader: Configuration loaded successfully, finalizing and returning
 INFO box_collection: Searching for box: aws-dummy (aws) in /Users/n0ts/.vagrant.d/boxes/aws-dummy/aws/metadata.json
 INFO box_collection: Box found: aws-dummy (aws)
 INFO environment: Applying 1 provider overrides. Reloading config.
 INFO loader: Set :"vm_test1_aws-dummy_aws" = [["2", #]]
DEBUG loader: Populating proc cache for ["2", #]
 INFO loader: Loading configuration in order: [:default, nil, :home, :root, :vm_test1, :"vm_test1_aws-dummy_aws"]
DEBUG loader: Loading from: default (cache)
DEBUG loader: Loading from: root (cache)
DEBUG loader: Loading from: vm_test1 (cache)
DEBUG loader: Loading from: vm_test1_aws-dummy_aws (evaluating)
DEBUG loader: Configuration loaded successfully, finalizing and returning
 INFO machine: Initializing machine: test1
 INFO machine:   - Provider: VagrantPlugins::AWS::Provider
 INFO machine:   - Box: #
 INFO machine:   - Data dir: //aws_test/.vagrant/machines/test1/aws
 INFO command: With machine: test1 (AWS (new))
 INFO interface: info: Bringing machine 'test1' up with 'aws' provider...
 INFO batch_action: Enabling parallelization by default.
 INFO batch_action: Batch action will parallelize: true
 INFO batch_action: Starting action: # up {:parallel=>true, :provision_enabled=>true, :provision_types=>nil, :provider=>"aws"}
 INFO machine: Calling action: up on provider AWS (new)
 INFO runner: Preparing hooks for middleware sequence...
 INFO runner: 3 hooks defined.
 INFO runner: Running action: #
 INFO warden: Calling action: #
 INFO handle_box_url: Skipping HandleBoxUrl because box or box_url not set.
 INFO warden: Calling action: #
 INFO warden: Calling action: #
 INFO warden: Calling action: #
 INFO warden: Calling action: #
 INFO warden: Calling action: #
 INFO warden: Calling action: #
 INFO warden: Calling action: #
 INFO connect_aws: Connecting to AWS...
 INFO warden: Calling action: #
 INFO runner: Preparing hooks for middleware sequence...
 INFO runner: 3 hooks defined.
 INFO runner: Running action: #
 INFO warden: Calling action: #
 INFO machine: Calling action: read_state on provider AWS (new)
 INFO runner: Preparing hooks for middleware sequence...
 INFO runner: 1 hooks defined.
 INFO runner: Running action: #
 INFO warden: Calling action: #
 INFO warden: Calling action: #
 INFO connect_aws: Connecting to AWS...
 INFO warden: Calling action: #
 INFO runner: Preparing hooks for middleware sequence...
 INFO runner: 3 hooks defined.
 INFO runner: Running action: #
 INFO warden: Calling action: #
 INFO warden: Calling action: #
 INFO warden: Calling action: #
 INFO warden: Calling action: #
 INFO interface: info: Updating Vagrant's berkshelf: '/Users/n0ts/.berkshelf/test1/vagrant/berkshelf-20130928-71302-17wcpxs-test1'
 INFO interface: info: Using dmp_ex_entrance (1.0.0)
 INFO warden: Calling action: #
 INFO warden: Calling action: #
 INFO warden: Calling action: #
 INFO interface: warn: Warning! The AWS provider doesn't support any of the Vagrant
high-level network configurations (`config.vm.network`). They
will be silently ignored.
 INFO warden: Calling action: #
 INFO interface: warn: Warning! You're launching this instance into a VPC without an
elastic IP. Please verify you're properly connected to a VPN so
you can access this machine, otherwise Vagrant will not be able
to SSH into it.
 INFO interface: info: Launching an instance with the following settings...
 INFO interface: info:  -- Type: m1.small
 INFO interface: info:  -- AMI: ami-d9118cd8
 INFO interface: info:  -- Region: ap-northeast-1
 INFO interface: info:  -- Keypair: default
 INFO interface: info:  -- Subnet ID: subnet-xxxx
 INFO interface: info:  -- Security Groups: ["sg-xxxx"]
 INFO interface: info:  -- Block Device Mapping: [{:DeviceName=>"/dev/sdf", :Ebs=>{:DeleteOnTermination=>false, :VolumeSize=>100, :VolumeType=>"standard"}}]
 INFO interface: info:  -- Terminate On Shutdown: false
ERROR warden: Error occurred: There was an error talking to AWS. The error message is shown
below:
Error: Expected(200) <=> Actual(400 Bad Request)
Response: 
InvalidRequestThe request received was invalid.2c2ce8f2-82b4-463d-bbed-5c875560eb99
 INFO warden: Beginning recovery process...
 INFO warden: Calling recover: #
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
ERROR warden: Error occurred: There was an error talking to AWS. The error message is shown
below:
Error: Expected(200) <=> Actual(400 Bad Request)
Response: 
InvalidRequestThe request received was invalid.2c2ce8f2-82b4-463d-bbed-5c875560eb99
 INFO warden: Beginning recovery process...
 INFO warden: Calling recover: #
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO environment: Running hook: environment_unload
 INFO runner: Preparing hooks for middleware sequence...
 INFO runner: 1 hooks defined.
 INFO runner: Running action: #
ERROR vagrant: Vagrant experienced an error! Details:
ERROR vagrant: # Actual(400 Bad Request)
Response: 
InvalidRequestThe request received was invalid.2c2ce8f2-82b4-463d-bbed-5c875560eb99>
ERROR vagrant: An error occurred while executing multiple actions in parallel.
Any errors that occurred are shown below.
An error occurred while executing the action on the 'test1'
machine. Please handle this error then try again:
There was an error talking to AWS. The error message is shown
below:
Error: Expected(200) <=> Actual(400 Bad Request)
Response: 
InvalidRequestThe request received was invalid.2c2ce8f2-82b4-463d-bbed-5c875560eb99
ERROR vagrant: /Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/lib/vagrant/batch_action.rb:109:in `run'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/lib/vagrant/environment.rb:210:in `block (2 levels) in batch'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/lib/vagrant/environment.rb:205:in `tap'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/lib/vagrant/environment.rb:205:in `block in batch'
:10:in `synchronize'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/lib/vagrant/environment.rb:204:in `batch'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/plugins/commands/up/command.rb:41:in `execute'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/lib/vagrant/cli.rb:46:in `execute'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/lib/vagrant/environment.rb:478:in `cli'
/Applications/Vagrant/embedded/gems/gems/vagrant-1.2.7/bin/vagrant:84:in `'
/Applications/Vagrant/bin/../embedded/gems/bin/vagrant:23:in `load'
/Applications/Vagrant/bin/../embedded/gems/bin/vagrant:23:in `'
 INFO interface: error: An error occurred while executing multiple actions in parallel.
Any errors that occurred are shown below.
An error occurred while executing the action on the 'test1'
machine. Please handle this error then try again:
There was an error talking to AWS. The error message is shown
below:
Error: Expected(200) <=> Actual(400 Bad Request)
Response: 
InvalidRequestThe request received was invalid.2c2ce8f2-82b4-463d-bbed-5c875560eb99
[31mAn error occurred while executing multiple actions in parallel.
Any errors that occurred are shown below.
An error occurred while executing the action on the 'test1'
machine. Please handle this error then try again:
There was an error talking to AWS. The error message is shown
below:
Error: Expected(200) <=> Actual(400 Bad Request)
Response: 
InvalidRequestThe request received was invalid.2c2ce8f2-82b4-463d-bbed-5c875560eb99[0m

pdeschen commented 11 years ago

+1 having the same issue

tralamazza commented 10 years ago

@pdeschen could you also post your debug log Thank you

tralamazza commented 10 years ago

Please remove the :Ebs => object, see this example https://github.com/mitchellh/vagrant-aws/issues/119#issuecomment-23786612

mitchellh / vagrant-aws

Feature Request: support attaching EBS volumes #38