search

hashicorp / packer

Packer is a tool for creating identical machine images for multiple platforms from a single source configuration.
http://www.packer.io
Other
15.12k stars 3.33k forks source link

Directory copy with file provisioner in windows does not copy files #9060

Open carrickhines opened 4 years ago

carrickhines commented 4 years ago

Overview of the Issue

When using the file provisioner in windows, the file provisoner does not copy files.

Reproduction Steps

{
"type": "file",
"source": "{{user `WorkingDirectory`}}\\",
"destination": "C:\\temp"
},

Packer version

From 1.5.5

Simplified Packer Buildfile

The file provisioner above is really the only thing required.

Operating system and Environment details

Windows, Azure, Azure Devops build pipeline

Log Fragments and crash.log files

The log shows the copy successfully completing but the files are not present in the destination directory.

nywilken commented 4 years ago

Hi @carrickhines thanks for reaching out. Can you please provide a copy of your debug logs PACKER_LOG=1 packer build build.json along with a sample of the value being used for the WorkingDirectory variable. I quickly tested the file provisioner on AWS with a windows AMI and was not able to reproduce the issue

amazon-ebs: output will be in this color.

==> amazon-ebs: Prevalidating any provided VPC information
==> amazon-ebs: Prevalidating AMI Name: packer-demo-1586882098
    amazon-ebs: Found Image ID: ami-02fc0cb4aa47ce2e9
==> amazon-ebs: Creating temporary keypair: packer_5e95e632-66b1-05e5-5f24-fb9ed8d51af1
==> amazon-ebs: Creating temporary security group for this instance: packer_5e95e639-2cdd-c503-28aa-196ebf16fbfe
==> amazon-ebs: Authorizing access to port 5985 from [0.0.0.0/0] in the temporary security groups...
==> amazon-ebs: Launching a source AWS instance...
==> amazon-ebs: Adding tags to source instance
    amazon-ebs: Adding tag: "Name": "Packer Builder"
    amazon-ebs: Instance ID: i-0002c6b6c50058a5f
==> amazon-ebs: Waiting for instance (i-0002c6b6c50058a5f) to become ready...
==> amazon-ebs: Skipping waiting for password since WinRM password set...
==> amazon-ebs: Using winrm communicator to connect: 184.72.193.212
==> amazon-ebs: Waiting for WinRM to become available...
    amazon-ebs: WinRM connected.
==> amazon-ebs: Connected to WinRM!
==> amazon-ebs: Uploading packertest => C:\temp
==> amazon-ebs: Provisioning with Powershell...
==> amazon-ebs: Provisioning with powershell script: /tmp/powershell-provisioner729964430
    amazon-ebs:
    amazon-ebs:
    amazon-ebs:     Directory: C:\Temp\packertest
    amazon-ebs:
    amazon-ebs:
    amazon-ebs: Mode                LastWriteTime     Length Name
    amazon-ebs: ----                -------------     ------ ----
    amazon-ebs: -a---         4/14/2020   4:37 PM        172 build.json
==> amazon-ebs: Pausing at breakpoint provisioner.
==> amazon-ebs: Press enter to continue.

By chance does is the path of WorkingDirectory on a different drive, maybe some network share?

I remember seeing a previous issue where a user had issues copying from a mapped drive. Here's the link to that issue https://github.com/hashicorp/packer/issues/8758

carrickhines commented 4 years ago

Hey @nywilken. I do have a few updates:

` Output of "Get-ChildItem C:\packages\CaC\ | Write-Output"

azure-arm: LastWriteTime : 4/14/2020 7:44:19 PM

azure-arm: Length        : 211

azure-arm: Name          : baseline.Tests.ps1

azure-arm:

azure-arm:
azure-arm: LastWriteTime : 4/14/2020 7:44:22 PM

azure-arm: Length        : 16436

azure-arm: Name          : localhost.mof

` I inserted a "Get-ChildItem C:\packages\ | Write-Output" into a powershell provisioner to return the contents of packages. After seeing that it was copying CaC (the source dir) I then inserted "Get-ChildItem C:\packages\CaC\ | Write-Output". The output is what you see above. That is, the contents of the source directory is in fact there. Here's why I thought provisioner was not copying files and why I think it is a bug:

From: https://www.packer.io/docs/provisioners/file.html

"If the source, however, is /foo/ (a trailing slash is present), and the destination is /tmp, then the contents of /foo will be uploaded into /tmp directly."

My assumption was that the contents of the source directory would be copied into c:\packages. That is not the behavior I am experiencing. The behavior I see is described in the proceeding file provisioner section of the docs as the following:

"If the source is /foo (no trailing slash), and the destination is /tmp, then the contents of /foo on the local machine will be uploaded to /tmp/foo on the remote machine. The foo directory on the remote machine will be created by Packer."

In the reproduction steps above, my example does have a trailing slash. I'm ready to chalk this up as a peculiarity in Windows. Perhaps I am also missing something in the documentation.

Thoughts?

Also, I have updated the repo steps to fit the output provided.

nywilken commented 4 years ago

@carrickhines thanks for the update - super helpful. This looks like a bug for sure. In reading the code I see that on Windows it is checking to see if the path ends with a Unix style path separator / and not the Windows path separator \. If you change your example to the following it should work as expected for now. 

{
"type": "file",
"source": "{{user `WorkingDirectory`}}/",
"destination": "C:\\temp"
},

Below are the results of my testing using the following config

{
"type": "file",
"source": "packertest/",
"destination": "c:\\Windows\Temp"
},

### results of copying the file contents, which is just build.json
==> amazon-ebs: Connected to WinRM!
==> amazon-ebs: Uploading packertest/ => c:\Windows\Temp
==> amazon-ebs: Provisioning with Powershell...
==> amazon-ebs: Provisioning with powershell script: /tmp/powershell-provisioner013559557
    amazon-ebs:
    amazon-ebs:
    amazon-ebs:     Directory: C:\Windows\Temp
    amazon-ebs:
    amazon-ebs:
    amazon-ebs: Mode                LastWriteTime     Length Name
    amazon-ebs: ----                -------------     ------ ----
    amazon-ebs: d----         4/16/2020  10:20 AM            0n0d0j51
    amazon-ebs: d----         3/11/2020   7:53 AM            AWSUpdateWindowsInstance
    amazon-ebs: d----         4/16/2020   6:48 AM            htwwcfcg
    amazon-ebs: -a---         3/10/2020  11:11 PM     161440 AWSUpdateWindowsInstance_1_4_4_0.z
    amazon-ebs:                                              ip
    amazon-ebs: -a---         4/16/2020  10:20 AM        172 build.json
rjnienaber commented 3 years ago

Based on this comment on #9386, it seems the resolution for this issue is to update the documentation to explicitly state you need a trailing forward slash, regardless of OS.