Closed dfrsol closed 5 years ago
SwampDragons
commented
6 years ago Hmm. I wonder if version 10 has changed the path to the leases file. Any chance you can kick off a build and poke around in your OS somewhere around /Library/Preferences/VMware Fusion/vmnet0/ and see if you can see something named dhcpd.conf or dhcp.conf that seems relevant to us here?
Meanwhile I'll try to track down version 10 myself.
SwampDragons
commented
6 years ago I'll also take a look at the differences between 1.2.2. and the current one.
SwampDragons
commented
6 years ago This is weird. The pathing for this file hasn't changed inside of Packer since before v 1.2.2.
azr
commented
6 years ago Hello, using Packer v1.3.2 and VMWare Fusion Professional Version 10.1.3 (9472307), it worked for me; I first built an ubuntu vmx image and was able to ssh to it.
I also built a centos 7 vmx image from bento and could also ssh to it.
❯ ls /Library/Preferences/VMware\ Fusion/
lastLocationUsed networking vmnet1
license-fusion-100-e3-201704 thnuclnt vmnet8Edit: trying to think of reasons 🙂
SwampDragons
commented
6 years ago Is there any more information you can give us on this one? We're at a loss because the relevant code doesn't seem to have changed between versions, and we can't reproduce.
dfrsol
commented
6 years ago Thanks for looking into this, the only thing I can think of is that they're using the trial version of VMware Fusion Pro. I'll attempt to use a base ubuntu vmx image as @azr has done above and see what happens.
azr
commented
6 years ago Hello @dfrsol, did it work ? 🙂
dfrsol
commented
6 years ago Hey @azr, sorry for the delayed response I'm just getting back from vacation. I'll reach out to my guys to see if this worked.
mvermaes
commented
6 years ago Just as another data point, I have the same issue here:
centos/7 Vagrant box)Packer 1.2.1 was the last version that didn't have the issue.
Using the same Packer template, but with the bento/centos-7 box, progresses past this point. So it seems like it might be caused by some interaction with this specific source image (in my case at least). I couldn't see anything obvious in the history of the source box's kickstart that would change this behaviour, but I tried using an older version of the centos/7 box anyway (1801.02 from the start of this year), with the same result.
SwampDragons
commented
6 years ago @mvermaes can you share your full debug logs? As far as I can see, the code in v1.2.1 is the same a the code in use today.
mvermaes
commented
6 years ago Sorry, 1.2.3 is where it first started happening - 1.2.2 is OK.
Logs uploaded to https://gist.github.com/mvermaes/f01e03f4b2ef3b78cde01b63f7868518
It seems to be due to the interface being assigned to the VMware bridged network instead of NAT. The source vmx for the centos/7 box has an ethernet0 device defined in it:
ethernet0.present = "TRUE"
ethernet0.virtualDev = "e1000"
ethernet0.connectionType = "bridged"but the Bento box does not. Changing the source vmx to use ethernet0.connectionType = "nat" makes everything work under 1.3.1.
Maybe this is related to 72da7cbf (case-insensitivity of the connectionType option)? I have "ethernet0.connectiontype": "nat" in the Packer config, but changing this to connectionType, or changing the source vmx to match the Packer config, made no difference.
mvermaes
commented
6 years ago Reading a bit more, it looks like prior to 72da7cb, connectiontype was being set to nat by default because the value in the vmx was not being matched due to the difference in case. I ran the build with -debug and checked the output vmx prior to it being cleaned up, and ethernet0.connectiontype = "nat" is present as expected.
I think it might have something to do with vmnetwork being set to "bridged" in the state variable here:
https://github.com/hashicorp/packer/blob/fb1be8be30880cebb8dd15f2287fa929753bd8d1/builder/vmware/vmx/step_clone_vmx.go#L101
but not being updated to have the "nat" value by the time this runs:
https://github.com/hashicorp/packer/blob/a26106055cb83f8bd4c4cdfca28b462d6a4db955/builder/vmware/common/driver.go#L501-L503
SwampDragons
commented
6 years ago @arizvisa I think at this point you're more familiar with the vmware code than me. Any insights?
arizvisa
commented
6 years ago The case-sensitivity patch was because another committer did some work (while the PR was being reviewed) which switched .vmx interaction to use ReadVMX which allowed users to interact with the file format as an associative array. EncodeVMX called strings.ToLower and so if one wanted to set a field in the .vmx, it needed to be in lowercase form. Whenever accessing this .vmx array, everything _needs_ to be in lowercase.
So the missing file path is simply that, the file was not found. By default (to remain backwards compatible with previous versions of templates) it falls back to "nat" since versions of packer prior to that assumed nat for everything (because previous versions of packer didn't actually support any of the other network types). So even if you don't specify a network type or it can't figure out what you're trying to tell it, it should follow the same semantics as earlier versions of packer.
However, using "vmxdata" to explicitly set network options or disk devices is the _wrong\ way to specify any of those things because packer doesn't have any knowledge of the meaning of vmx_data. vmx_data is a hack because packer is unable to "undo" any changes that are made by vmx_data. If a user specifies a custom vmx_data that conflicts with what packer is trying to do, then packer will act differently than one might expect. This is because packer doesn't parse 100% of the .vmx file specification (and assumes the rest of the file is opaque). This results in packer not being able to always figure out what to revert when going through its different steps. Again, packer only really supports the explicit configuration options that are exposed because its been explicitly developed with knowledge on how to revert those things (like floppy disks and other devices, etc).
Generally it's a good habit to avoid using vmx_data, and if some .vmx config isn't supported by packer then it warrants a creation of a PR to add support for said feature. Anyways, the correct way to specify a network type is to use "network" : "nat", or "network" : "hostonly" or "network": "vmnetX" as doc'd at https://packer.io/docs/builders/vmware-iso.html#network. Same thing for disk device types, or any other of those custom things.
arizvisa
commented
6 years ago Also after looking at how vmx_data and vmx_data_post is being used in this issue to change the hardware configuration while building and after the template is built, it seems like this isn't how packer was first written. I'd personally consider this type of hardware re-configuration as part of the provisioning process before deployment, but that's a matter of opinion really.
Although vmx_data_post is useful, I don't think any of the other builders support these types of semantics. Normally packer is used for building the base hardware configuration and OS install, and then when deploying with PowerCLI or Terraform or vagrant or whatever, then you'll tweak any aspects specific to the deployed VM. In this case, vmx_data_post is being used to sort of "lock-in" a specific configuration.
I think it's important to distinguish this in that packer is used for building the base template with an OS, and is to be used combined with another tool in order to add specific hardware configurations and things before deployment. I had a hard time distinguishing that at first as other devops tools were at the point of maturing, and I just wanted to use packer for everything. Although maybe vmx_data_post should not be part of the "builder" phase, and actually should be part of the "provision" or "post-processors" phase.
But, using it to switch to one hardware configuration for doing the os install, and then another configuration before deployment is using it in a way that it wasn't originally written for (since none of the other builders have started out supporting those configuration options).
But yeah, again...just a matter of opinion.
SwampDragons
commented
6 years ago @dfrsol if you launch a build and look in "/Library/Preferences/VMware Fusion/" are there any directories that contain a "dhcpd.conf" or similarly named file?
arizvisa
commented
6 years ago @SwampDragons, I made an example PR #6856 which takes the functionality of vmx_data_post and adds it as a post-processor. Like really, a post-processor makes more sense since Packer should only be focused on building the template and performing OS install, not changing the hardware configuration prior to deployment. It'll need some work, but it should help prevent these types of issues from happening.
Like afaict, the reason why dhcpd.conf is being searched for is because Packer is trying to determine the host/network to connect to. This is because "vmx_data" is being used to switch the network connection type and packer removes any options that begin with "ethernet*".
If a user tampers with any of them, then the user might actually prevent packer from reverting them. The vmware builder (and a few others) remove all the ethernet interfaces and floppy disks before actually exporting it. So, using vmx_data_post to try and force certain hw configurations will conflict and only cause more of these types of bugs to occur.
Maybe someone should add to the documentation for vmx_data that it is to be used _secondary_ to the configuration options that Packer exposes and that the "builder" step is strictly for configuring the VM in order to install the operating system (not for specifying the hardware configuration for the exported template). I would, but my regular job doesn't involve coding or devops or any of this stuff so it's hard for me to find projects that are parallel to this stuff.
SwampDragons
commented
6 years ago Thanks! I'll look at the PR and at making the docs clearer. Maybe we can add some kind of an override warning during validation, where we check the vmx data for dangerous-to-override values that could cause Packer to break.
arizvisa
commented
6 years ago @SwampDragons, I'll leave it open as a PR and give you access to the branch since it's really more of an example of shifting these vmx_data issues to the correct place (there's been a couple of issues related to vmx_data being used instead of using the configuration options available in the vmware builders).
Users have been mis-using vmx_data and vmx_data_post for things like setting the hardware version for example, network settings, hard disk type, etc. Again, each of these things packer has explicit knowledge of unless vmx_data is used.
But yeah, don't actually merge that PR. Especially not until you get a chance to take a look at it and tweak it. I'm not even sure if it actually works or if I'm doing it the correct way. In the post-processor hook, I walk through all of the files associated with an artifact and then modify it directly. I wasn't sure if I was supposed to make a copy of the artifact's file and give it the same name, or it it's even okay to modify directly. I think a post-processor will also need to be made for virtualbox as well. But feel free to close the PR whenever you're done with it or if you lose interest ;).
I think drawing this line between building and post-processing will prevent a lot of issues in the future and is important enough so that devers can avoid having to parse all the things available within a .vmx file (since the meaning of some parts of .vmx can change depending on the VMware version).
SwampDragons
commented
6 years ago Awesome, thanks.
SwampDragons
commented
6 years ago @dfrsol did any of the suggestions on this issue help you solve your problem?
SwampDragons
commented
5 years ago I'm going to close this since we've been waiting for a reply for over a month. If anyone is still experiencing this and can help us reproduce, we can reopen.
ghost
commented
4 years ago I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.
Packer:
>= 1.2.5Host: MacOS High Sierra 10.13.4 VMware: Fusion 10.1.3We're currently on-boarding new team members and told them to download the latest version of Packer, but unfortently we're currently running into a blocking issue when using VMware Fusion 10.1.3 with any version of Packer
>= 1.2.5to build OVAs.This error does not occur when using Packer version
1.2.2.Error detecting host IP: Could not find vmnetdhcp conf file: /Library/Preferences/VMware Fusion/vmnet0/dhcpd.conf
[1;32m==> vmware-vmx: Cloning source VM...[0m [1;32m==> vmware-vmx: Starting virtual machine...[0m [1;32m==> vmware-vmx: Waiting 10s for boot...[0m [1;32m==> vmware-vmx: Connecting to VM via VNC (127.0.0.1:5940)[0m [1;31m==> vmware-vmx: Error detecting host IP: Could not find vmnetdhcp conf file: /Library/Preferences/VMware Fusion/vmnet0/dhcpd.conf[0m [1;32m==> vmware-vmx: Stopping virtual machine...[0m [1;32m==> vmware-vmx: Deleting output directory...[0m [1;31mBuild 'vmware-vmx' errored: Error detecting host IP: Could not find vmnetdhcp conf file: /Library/Preferences/VMware Fusion/vmnet0/dhcpd.conf[0m
==> Some builds didn't complete successfully and had errors: --> vmware-vmx: Error detecting host IP: Could not find vmnetdhcp conf file: /Library/Preferences/VMware Fusion/vmnet0/dhcpd.conf
==> Builds finished but no artifacts were created.
Gist for build:
https://gist.github.com/dfrsol/fad4672d2624fd7d5c6daa9a903c3e7b
Gist for template (ssh user and pass removed):
https://gist.github.com/dfrsol/5e76aba3fd0c1a419654773ccae858aa