Windows Server 2016

[matthargett]

Right now, we can build our Universal Windows App, but can't run any tests on AppVeyor.

Please add a Windows Server 2016 environment, now that RTM build have been available for a month.

(Note: We use shared code between our WPF and UWP projects, so we still need the older OS environments available as well.)

Updating Docker:

Install-Package -Name docker -ProviderName DockerMsftProvider -Verbose -Update -Force
start-service docker

[IlyaFinkelshteyn]

@friism @Yshayy Added!

[friism]

@IlyaFinkelshteyn @FeodorFitsner do you think there's going to be any way to cache Docker layers between builds? The simplest way would be to cache $Env:ProgramData\docker, but that vastly exceeds the 100MB that's the documented build cache limit.

Having to rebuild from scratch sucks

[FeodorFitsner]

Cache is 1 GB for free accounts. 100 MB was an old limits. Give it a try and let me know how it worked.

[friism]

@FeodorFitsner ah, ok - there's a mention of 100MB here, but I see there's an updated table below.

1GB is not enough either for Windows containers (eg. the base layers alone 10s of GB). I don't think larger cache limits are a reasonable request, and I'm pondering a more reasonable alternative.

[Yshayy]

Hi, is there an image version with support for both V2017RC and docker?

[FeodorFitsner]

Combined Windows 2016 + VS 2017 image will be available shortly after VS 2017 release.

[Yshayy]

Is there any workaround to build VS2017 aspnet core with docker deployment on Appveyor?

[friism]

@Yshayy You can just use Docker: https://github.com/friism/MusicStore/blob/master/Dockerfile.windows#L1

That Dockerfile builds a ASP.NET Core app. This docker-compose file runs it with SQL Server: https://github.com/friism/MusicStore/blob/master/docker-compose.windows.yml

And AppVeyor runs it: https://github.com/friism/MusicStore/blob/master/appveyor.yml#L8

You don't need it to do the above, but here's a Dockerfile with the VS2017 build tools: https://github.com/friism/dockerfiles/blob/master/vs-build-tools/17/Dockerfile

[hferentschik]

I would be interested in testing the Windows 2016 image as well. My account is 'hferentschik'. My use case would be to run nested virtualization. Any idea whether this would work. We are working on a Go project which creates and manages a VM usings Docker's libmachine. We can already build and unit test on AppVeyor, but to be able to run our integration tests we would need nested virtualization.

[StefanScherer]

@hferentschik Nested virtualization would be great, but I don't think it will work for 64bit machines. But you can read https://stefanscherer.github.io/how-to-create-a-32bit-boot2docker-for-x86-cpus/, about a year ago I ran a 32bit docker machine in AppVeyor. Maybe that helps you a little bit.

[gbraad]

@FeodorFitsner, I work with @hferentschik on the same project. It would be great to have access to try some stuff. account name is 'gbraad'.

[IlyaFinkelshteyn]

@hferentschik @gbraad Added!

[bsideup]

Hi!

I'm co-maintainer of http://github.com/testcontainers/testcontainers-java and we would like to start testing it on Windows.

Could you please add my account (I use fork to develop) and testcontainers org to the beta program?

Thanks!

[IlyaFinkelshteyn]

@bsideup Sure, added.

[bsideup]

@IlyaFinkelshteyn thank you!

[smudge202]

Could I also be added please, @IlyaFinkelshteyn ? <3

[IlyaFinkelshteyn]

@smudge202 Sure, added!

[smudge202]

EDIT: Solved, see bottom.

Is anyone else having problems with volume mounting? I can't for love nor money get APPVEYOR_BUILD_FOLDER mounted despite following workarounds suggested here:

• https://github.com/docker/docker/issues/12590#issuecomment-96767796
• http://superuser.com/questions/1051520/docker-windows-container-how-to-mount-a-host-folder-as-data-volume-on-windows

Example command:

$originalMountSpecification = "$(Get-Location)/:/app"
$mountSpecification = $originalMountSpecification.ToLowerInvariant().Replace("\", "/")
"'$originalMountSpecification' -> '$mountSpecification'"
# ^ Displays 'C:\projects\clean-space/:/app' -> 'c:/projects/clean-space/:/app'"
docker run -it --rm -v ${mountSpecification} --workdir /app microsoft/aspnetcore-build bash -c "dotnet restore CleanSpace.sln && dotnet publish src/api/api.csproj -c Release -o ./obj/Docker/publish"

Results in error:

docker : C:\Program Files\Docker\docker.exe: Error response from daemon: invalid bind mount spec "c:/projects/clean-space/:/app": invalid volume specification: 'c:\projects\clean-space\:\app'.
At C:\projects\clean-space\build\build-api-appveyor.ps1:12 char:1
+ docker run -it --rm -v "${mountSpecification}" --workdir /app microso ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (C:\Program File...n-space\:\app'.:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

Does Windows Server 2016 require config akin the Docker for Windows shared drives? If so, can this be done on the image or does someone know how to do it on the fly?

NB: I've tried more than 15 iterations of this command to try and get the syntax correct... thus beginning to assume this is a config issue? I'll spin up a Win Server 2016 vm in azure and see if I get the same issue there. Will update.

I believe it's because I'm trying to mount outside of the user directory. From docker documentation:

If you are using Docker Machine on Mac or Windows, your Docker Engine daemon has only limited access to your macOS or Windows filesystem. Docker Machine tries to auto-share your /Users (macOS) or C:\Users (Windows) directory.

I can't find any evidence that this remains true for containers in Windows Server 2016. ~Still waiting for aforementioned vm to test.~ Have confirmed this issue occurs on the VM, regardless of whether I'm in userprofile or not.

---

SOLUTION:

As a hacky workaround to bypass the path conversion cooked into windows (I believe it's something to do with mysys) you can skip the target binding (which implicitly sets the target to the same name as the leaf directory in the source). i.e.

docker run -it --rm -v "$(Get-Location)" --workdir "/$targetBind" microsoft/aspnetcore-build bash -c "dotnet restore CleanSpace.sln && dotnet publish src/api/api.csproj -c Release -o ./obj/Docker/publish"

.. will, in the case shown above, bind to a directory called "clean-space" in the container. You can use Split-Path (Get-Location) -Leaf to get the $targetBind shown in the command.

I've seen advice on disabling MSYS_NO_PATHCONV but not sure how to do it PS - I appreciate this may not be configured in the image by default but something in the docs on how to do it would be a god-send! Will leave this all here in case anyone hits a similar issue.

[FeodorFitsner]

@smudge202 I believe correct mount specification for Docker with Windows Containers might be the following:

$originalMountSpecification = "$(Get-Location)/:c:/app"

[smudge202]

The above was a linux image; I've switched to a windows image since (microsoft/dotnet:sdk-nanoserver). - for some reason I had it in my head that they'd have gotten round to enabing linux images on Win2k16, but this is obviously not the case).

The windows image however also has issues with mapping because the symlinkd doesn't work correctly for volume mounting unless you specify the full path. The latest iteration of the above command is:

docker run --rm -v "$(Get-Location):$(Get-Location)" --workdir "$(Get-Location)" microsoft/dotnet:sdk-nanoserver "dotnet"

Unfortunately this errors with:

C:\Program Files\Docker\docker.exe: Error response from daemon: container f84ce8779894e97766115eb9080d5b6cd094aa8a95bf7f4b2124787f15125427 encountered an error during CreateProcess: failure in a Windows system call: The system cannot find the file specified. (0x2) extra info: {"ApplicationName":"","CommandLine":"-c dotnet","WorkingDirectory":"C:\\projects\\clean-space","Environment":{"DOTNET_SDK_DOWNLOAD_URL":"https://dotnetcli.blob.core.windows.net/dotnet/Sdk/1.0.1/dotnet-de
v-win-x64.1.0.1.zip","DOTNET_SDK_VERSION":"1.0.1","NUGET_XMLDOC_MODE":"skip"},"EmulateConsole":false,"CreateStdInPipe":t
rue,"CreateStdOutPipe":true,"CreateStdErrPipe":true,"ConsoleSize":[0,0]}.

Utterly confusing because if I run the above in interactive (-it) without the command, I can run the dotnet restore/publish without issue, and the output is correctly available to the host. From what I've seen in vaguely related issues, I think my command syntax might be slightly out (for example, if you have options in the wrong place you also get this issue). However, I can't see how my command differs from the docker documentation.

EDIT: Nevermind... removing the quotes from the command seems to do the job. The next problem was running two commands on one line... (& not allowed when not quoted) so I moved the 2 lines to a cmd file and just call that instead. Last but not least... seeing as you're here, @FeodorFitsner , I have $WarningPreference = 'SilentlyContinue' set in the build script but they still show as errors in appveyor. Couldn't see anything in the docs to prevent that (though I am going cross-eyed at this stage).

[StefanScherer]

@smudge202 At the moment it's a little bit difficult to see if its a linux or windows image. I also had to look at it with docker run stefanscherer/winspector microsoft/aspnetcore-build first and it showed me that it's not a windows image.

I tried your command in an empty folder on the host and it worked for me. Probably it depends in which folder you are on the host as you can't map to a non-empty folder in the container. So the second Get-Location is not needed, use a fix folder inside the container to get rid of that dependency.

Use eg.

docker run --rm -v "$(Get-Location):C:\work" --workdir "C:\work" microsoft/dotnet:sdk-nanoserver dotnet

[smudge202]

Yup, so long as I specify a drive in the target bind it should be right. I tend to just hop into docker hub and walk down the dockerfile FROM's until it shows nano/win. If it's not one of those, I assume it to be a linux distro. Like you say though, not obvious.

I did get everything working in the end (in that appveyor produces nanoserver based images for my aspnetcore stuff). I don't like that the warning docker produces when it can't find an image locally causes all remaining appveyor build output to show as errored text, but it does show as build success (tips on that greatly appreciated).

For all that effort, my end goal is to get images into my Docker Enterprise swarm in azure, which is currently linux based. Off to TravisCI for that half of things it seems (I don't believe any of the appveyor environments are able to produce linux based docker images?).

[hpryce]

Hi,

Is it possible to enable the image for https://github.com/palantir/docker-compose-rule so we can use it to test compatibility with Windows?

Thanks!

[IlyaFinkelshteyn]

@hpryce If I understood correctly, AppVeyor account name is iamdanfox. Added.

[IlyaFinkelshteyn]

The new Visual Studio 2017 build worker image is now available. It has Visual Studio Community 2017 RTM installed on Windows Server 2016 with Docker and base Docker images.

Currently, the image is being run on Google Compute Engine (GCE) environment only, so builds will be starting in 3-4 minutes (time required to provision and boot up build VM). The image will be available on Hyper-V environment later this week.

[StefanScherer]

@IlyaFinkelshteyn Awesome! So all the private beta testers just have to change image: Windows Server 2016 into image: Visual Studio 2017. That's easy.

[FeodorFitsner]

yep!

[PeterMTaylor]

Just letting you know I got a successful run with using image from WMF 5 to Visual Studio 2017. Thanks for that.

[FeodorFitsner]

Windows Server 2016 image has been removed. Please adjust your builds to use Visual Studio 2017 image instead which is based on Windows Server 2016 with Docker installed.