search

nsidc / vagrant-vsphere

VMware vSphere provider for Vagrant
Other
608 stars 166 forks source link

Parallel provisioning #210

Open taliesins opened 7 years ago

taliesins commented 7 years ago

With these changes I have managed to spin up 10 Windows boxes in parallel. I also make use of a few vagrant plugins like: vagrant-berkshelf, vagrant-triggers and vagrant-windows-domain. So there is lots of crazy things happening like reboots and re-initialization of file shares.

This PR is going to need some help on the test side. I am a bit of a Ruby noob, so I am not sure how we should be changing the tests to suite the code changes. So the tests have been left unchanged. Please send me a PR to fix the tests!

ghost commented 7 years ago

@taliesins I merged your PR #204 (by way of #212 where I fixed tests, etc), but that has caused the parallel-provisioning branch to have some rather large merge conflicts, particularly in lib/vSphere/action/clone.rb. Would you be able to take care of resolving those conflicts?

taliesins commented 7 years ago

@michael-brandt-cu I have resolved the conflicts. It would be wonderful if you could help me with the tests. Would love to get this into the main branch.

I have added a couple more things into this pull request:

ghost commented 7 years ago

@taliesins

I have resolved the conflicts.

Great!

It would be wonderful if you could help me with the tests.

I've created a new branch, parallel-provisioning-tests, starting from what you have here, with changes to get rubocop and the tests passing so that the Travis build can pass. I don't have it all passing yet, but that's where I'm planning to continue work on that.

I have a 3-day weekend this weekend, but I'll be able to get back to this next week.

ghost commented 7 years ago

I took a break from fixing unit tests, and I tried using this branch to destroy and recreate a VM I use for development on my current main project at NSIDC. I got these errors when attempting to destroy it:

There are errors in the configuration of this machine. Please fix
the following errors and try again:

vSphere Provider:
* The following settings shouldn't exist: real_nic_ip, vlan

We use both of these settings in our own infrastructure, and it looks like this branch removes real_nic_ip and changes how vlan works. Considering these breaking changes, and the possibility that other configuration settings are also changed in a backwards-incompatible way, I'm hesitant to pull this in.

It might be possible to release these changes in their current form as vagrant-vsphere-parallel, but then we would quickly have two diverging plugins; given how many changes this PR has, any features/fixes added to vagrant-vsphere could be difficult to apply to vagrant-vsphere-parallel, and vice versa.

taliesins commented 7 years ago

@michael-brandt-cu real_nic_ip, vlan are configuration settings that used to be at machine level. To support multiple nics we need to move them to network adapter level.

Perhaps we could put in a shim property for vlan and ip_address?

It should be trivial to change your vagrant scripts to use the multiple network card approach:

Here is an example snippet from my vagrant file for setting up Juniper FCP for vMX:

      fpc.vm.provider :vsphere do |vsphere, overrides|
        vsphere.host = vsphere_host
        vsphere.insecure = true
        vsphere.user = vsphere_username
        vsphere.password = vsphere_password        
        vsphere.compute_resource_name = vsphere_cluster
        vsphere.resource_pool_name = vsphere_resource_pool   
        vsphere.template_name = vsphere_template_juniperfpc
        vsphere.name = fpc.vm.hostname
        vsphere.vm_base_path = "#{vsphere_vm_base_path}/Env#{environment_number}"
        vsphere.data_store_name = vsphere_datastore
        vsphere.memory_mb = 8192
        vsphere.cpu_count = 3
        vsphere.management_network_adapter_slot = 0
        vsphere.management_network_adapter_address_family = 'ipv4'
        vsphere.network_adapter 0, vlan: "Env#{environment_number}", mac_address: macAddress, ip_address: ipAddress
        vsphere.network_adapter 1, vlan: "Env#{environment_number}-Vmx#{i}"
        vsphere.network_adapter 2, vlan: "Env#{environment_number}-TransportBetweenVmx#{i % 2 == 1 ? i : i-1}AndVmx#{i % 2 == 0 ? i : i+1}"
        vsphere.network_adapter 3, vlan: "Env#{environment_number}-#{i % 2 == 1 ? 'Client' : 'CSP'}#{i % 2 == 1 ? i : i-1}-2"
        vsphere.network_adapter 4, vlan: "Env#{environment_number}-#{i % 2 == 1 ? 'Client' : 'CSP'}#{i % 2 == 1 ? i : i-1}-4"
        vsphere.network_adapter 5, vlan: "Env#{environment_number}-#{i % 2 == 1 ? 'Client' : 'CSP'}#{i % 2 == 1 ? i : i-1}-6"
        vsphere.network_adapter 6, vlan: "Env#{environment_number}-#{i % 2 == 1 ? 'Client' : 'CSP'}#{i % 2 == 1 ? i : i-1}-1"
        vsphere.network_adapter 7, vlan: "Env#{environment_number}-#{i % 2 == 1 ? 'Client' : 'CSP'}#{i % 2 == 1 ? i : i-1}-3"
        vsphere.network_adapter 8, vlan: "Env#{environment_number}-#{i % 2 == 1 ? 'Client' : 'CSP'}#{i % 2 == 1 ? i : i-1}-5"
        vsphere.network_adapter 9, vlan: "Env#{environment_number}-#{i % 2 == 1 ? 'Client' : 'CSP'}#{i % 2 == 1 ? i : i-1}-7"
      end
macster84 commented 6 years ago

Hi everyone,

looks like there is lot of potential in this pull request. Any chance we can get this or parts of it merged?

@michael-brandt-cu: Is backwards compatibility your main concern?

Maybe we can split the pull request in multiple different ones so that risk and merge effort is reduced? In any case, I would be happy to help.

ghost commented 6 years ago

@michael-brandt-cu: Is backwards compatibility your main concern?

Yes.

Maybe we can split the pull request in multiple different ones so that risk and merge effort is reduced?

Maybe. It's been a while since I've had funding to do much work on vagrant-vsphere, and even longer since I really looked at this PR, so I can't really say how well it could be split up.