search

test-kitchen / kitchen-vra

A Test Kitchen driver for VMware vRealize Automation
Apache License 2.0
13 stars 22 forks source link

Hostname vs IP #25

Closed BenMitchell1979 closed 6 years ago

BenMitchell1979 commented 6 years ago

Versions:

Platform Details

Scenario:

Deploying Windows server via Kitchen in a non-domain joined method. When the VM comes online - kitchen returns the name of the VM instead of the IP address. Since these are not being joined to the domain - the VM Name is not being added to my DNS dynamically. It really just needs to return the IP address of the server so that connection can always be established.

Steps to Reproduce:

driver: name: vra username: vralogin@domain.com password: mypassword tenant: vsphere.local base_url: https://vra.domain.com verify_ssl: false

transport: name: winrm username: Administrator password: mypassword

provisioner: name: chef_zero always_update_cookbooks: true

verifier: name: inspec

platforms:

suites:

Expected Result:

I expected kitchen-vra to return IP address vs VM name for my instance and establish connection.

Actual Result:

[core.cgnlabs.com]: PS C:\VMs\projects\chef-kitchen-work\kitchen-vra\vra-win2016-kitchen> chef exec kitchen verify -----> Starting Kitchen (v1.19.2) -----> Creating ... Building vRA catalog request... chef : >>>>>> ------Exception-------

Class: Kitchen::ActionFailed Message: 1 actions failed. Failed to complete #create action: [Unable to submit request: Invalid filter '(userName eq "sa_admin" and domain eq null)': Invalid filter value beginning at position 38] on default-windows2016

Please see .kitchen/logs/kitchen.log for more details Also try running kitchen diagnose --all for configuration [core.cgnlabs.com]: PS C:\VMs\projects\chef-kitchen-work\kitchen-vra\vra-win2016-kitchen> chef exec kitchen verify -----> Starting Kitchen (v1.19.2) -----> Creating ... Building vRA catalog request... Catalog request 4fc75f8f-ade4-4824-83f0-d3dc44fff9de submitted. Current request status: PENDING_PRE_APPROVAL Current request status: IN_PROGRESS Current request status: PENDING_POST_APPROVAL Waiting For vRA to collect the IP$$$$$$ Server acbbf919-a6af-4db6-8676-83c03f0294f3 has no IP address. Falling back to server name (cgn-028)... Server acbbf919-a6af-4db6-8676-83c03f0294f3 (cgn-028) created. Waiting until ready... $$$$$$ Server acbbf919-a6af-4db6-8676-83c03f0294f3 (cgn-028) not reachable: SocketError -- getaddrinfo: No such host is known. (cgn-028:5985) $$$$$$ Sleeping 5 seconds and retrying... $$$$$$ Server acbbf919-a6af-4db6-8676-83c03f0294f3 (cgn-028) not reachable: SocketError -- getaddrinfo: No such host is known. (cgn-028:5985) Retries exceeded. Destroying server... Destroy request eb7ce906-f473-45ab-a70f-aff7d867bed4 submitted. Current request status: PENDING_PRE_APPROVAL Current request status: IN_PROGRESS Current request status: PENDING_POST_APPROVAL Destroy request complete. chef : >>>>>> ------Exception-------

  • CategoryInfo : NotSpecified: (>>>>>> ------Exception-------:String) [], RemoteException
  • FullyQualifiedErrorId : NativeCommandError

Class: Kitchen::ActionFailed Message: 1 actions failed. Failed to complete #create action: [getaddrinfo: No such host is known. (cgn-028:5985)] on default-windows2016

Please see .kitchen/logs/kitchen.log for more details Also try running kitchen diagnose --all for configuration [core.cgnlabs.com]: PS C:\VMs\projects\chef-kitchen-work\kitchen-vra\vra-win2016-kitchen>

BenMitchell1979 commented 6 years ago

I needed to correct this Issue. It does not return the hostname - it returns the VM name (which is different than the hostname).

Also the results are not consistent every build. Sometimes the IP does get returned and the deployment succeeds and other times the VM Name gets returned.

jjasghar commented 6 years ago

Yep, the last time we looked at the API there was no way to get the hostname specifically, only what the vRA instance had in which was the vm name. This is a limitation of the vRA API as of 7.2-3, and unless someone knows something better I don't think this is possible.

cc: @SDBrett

BenMitchell1979 commented 6 years ago

Any idea why on some attempts it grabs the IP and works? I honestly think grabbing the IP everytime is the best way - hostnames might not make into DNS in a timely fashion.

SDBrett commented 6 years ago

Is the IP address assigned in the same way for each deployment? How do the VM's get their IP address? IPAM, DHCP or Static? There is a difference in tracking between the methods.

BenMitchell1979 commented 6 years ago

Same catalog item in vra. Windows 2016 pulling via DHCP. First attempt fails with VM Name, delete the .kitchen folder and run chef exec kitchen verify again - and this time it pulls the IP address on the next build.

BenMitchell1979 commented 6 years ago

I'm guessing based on this section of the code - what is happening is that vRA is not returning the IP address to kitchen-vra in a timely enough fashion and it's falling back to 'server.name'. 

def create(state)
        return if state[:resource_id]

        server = request_server
        state[:resource_id] = server.id
        state[:hostname]    = hostname_for(server)
        state[:ssh_key]     = config[:private_key_path] unless config[:private_key_path].nil?

        wait_for_server(state, server)
        info("Server #{server.id} (#{server.name}) ready.")
      end

      def hostname_for(server)
        if config[:use_dns]
          raise 'No server name returned for the vRA request' if server.name.nil?
          return config[:dns_suffix] ? "#{server.name}.#{config[:dns_suffix]}" : server.name
        end

        ip_address = server.ip_addresses.first
        if ip_address.nil?
          warn("Server #{server.id} has no IP address. Falling back to server name (#{server.name})...")
          server.name
        else
          ip_address
        end
      end
jjasghar commented 6 years ago

So vRA doesn't own the IP addresses, your DHCP server does. There is a process in vRA that is called Data Collection: https://youtu.be/9MZNYOXJnaw?t=13m59s that happens on a cadence. You might be hitting a race condition here in this process.

BenMitchell1979 commented 6 years ago

@jjasghar - vRA stores the IP it gets from the IP Provider (DHCP, InfoBlox, etc) as an object with the VM info. You can query it by the APIs - requires a little digging - but you can grab it out. Is the Kitchen plugin not getting the build information from the vRA API objects?

edit to include example url: http://www.vmtocloud.com/how-to-get-the-ip-address-of-a-vm-in-vrealize-automation-7-rest-api/

BenMitchell1979 commented 6 years ago

Data Collection happens at resource allocation. So if you assign new resources to your vRA deployment you need to either wait for or manually trigger 'Data Collection' so that you can then have access to those resources in the BluePrints.

jjasghar commented 6 years ago

It's looking for specific portions of the response back: https://github.com/chef-partners/vmware-vra-gem/blob/master/lib/vra/resource.rb#L177-L186

SDBrett commented 6 years ago

My suspicion is because the IP is assigned through dhcp at the guest OS level, there is a delay in vRA learning about it.

Whereas setting static during deployment or IPAM, the IP is a property.

I'm verifying if this is the case what can be done.

That may cause a delay in vra having the record.

I'm verifying if this could be the case

On Wed, 6 Dec 2017 at 4:56 am, Benjamin Mitchell notifications@github.com wrote:

@jjasghar https://github.com/jjasghar - vRA stores the IP it gets from the IP Provider (DHCP, InfoBlox, etc) as an object with the VM info. You can query it by the APIs - requires a little digging - but you can grab it out. I guess Is the Kitchen plugin not getting the build information from the vRA API objects?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/chef-partners/kitchen-vra/issues/25#issuecomment-349386786, or mute the thread https://github.com/notifications/unsubscribe-auth/AYUE-YUxiPnqQzTlt-ZuE40SOTeTvwhXks5s9YPPgaJpZM4Q1mfs .

BenMitchell1979 commented 6 years ago

@SDBrett - that was my guess as well. Thanks!

SDBrett commented 6 years ago

Just confirmed, when using DHCP in vRA. The IP address and hostname information will not be known by vRA until 2 things happen During the guest customization process, the networking details are passed to vCenter through VMtools. vRA pulls that information from vCenter during data collection.

So if the last data collection was before the VM has finished the guest customization stage, vRA will not know that information.

On Wed, Dec 6, 2017 at 5:03 AM, Benjamin Mitchell notifications@github.com wrote:

@SDBrett https://github.com/sdbrett - that was my guess as well. Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/chef-partners/kitchen-vra/issues/25#issuecomment-349388927, or mute the thread https://github.com/notifications/unsubscribe-auth/AYUE-deIWPU_lD-It2HtvVA8eMMViAWWks5s9YV8gaJpZM4Q1mfs .

BenMitchell1979 commented 6 years ago

Nicely done! So is there a way to enter a wait state for Data Collection to process (or trigger a data collection via API) before moving forward with grabbing the IP?

BenMitchell1979 commented 6 years ago

I'm not a Ruby guy - so forgive me if I'm asking rudimentary questions: Looking at the linked code from @jjasghar - I'm not sure why it's exiting the while loop before it gets the IP address. I don't see an escape clause that can be fulfilled except by pulling an IP address into either data_zero || data_one. I'm also confused as to why they pull the ip_address separate from the while loop?

lgustafson commented 6 years ago

I believe this is caused by an issue with vmware-vra-gem. See https://github.com/chef-partners/vmware-vra-gem/issues/65. The related PR seems to have fixed the issue for me consistently.

jjasghar commented 6 years ago

I have released the new gem, @BenMitchell1979 can you give this another shot and say 👍 or 👎 if it's fixed?

BenMitchell1979 commented 6 years ago

@jjasghar Awesome! I'll update my install and test it this weekend (Monday on the latest). Thanks for this!

BenMitchell1979 commented 6 years ago

@jjasghar Looks good. I pulled the latest version down and completed 6 different builds with no errors. I'd say this one is closed !

jjasghar commented 6 years ago

Great to hear, thanks to @lgustafson for the fix!