VS2017 Docker-Compose Project breaks build on command line

[billpratt]

Steps to reproduce

In Visual Studio 2017: File->New Project->ASP.NET Core Web Application Choose either Web API or Web Application Check "Enable Docker Support"

Go to command line and run

dotnet restore ./[NAME].sln
dotnet build ./[NAME].sln

Expected behavior

Restore and build work when a Docker-Compose project (.dcproj) is in the solution. Should work across all platforms that dotnet is supported

Actual behavior

error MSB4019: The imported project "C:\Program Files\dotnet\sdk\1.0.0\Sdks\Microsoft.Docker.Sdk\Sdk\Sdk.props" was not found. Confirm that the path in the declaration is correct, and that the file exists on disk.

PS C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10> dotnet --version
1.0.0

PS C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10> dotnet restore .\WebApplication10.sln
C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\docker-compose.dcproj : error MSB4019: The imported project "C:\Program Files\dotnet\sdk\1.0.0\Sdks\Microsoft.Docker.Sdk\Sdk\Sdk.props" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.
  Restoring packages for C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\WebApplication10\WebApplication10.csproj...
  Restoring packages for C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\WebApplication10\WebApplication10.csproj...
  Restore completed in 785.82 ms for C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\WebApplication10\WebApplication10.csproj.
  Lock file has not changed. Skipping lock file write. Path: C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\WebApplication10\obj\project.assets.json
  Restore completed in 1.08 sec for C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\WebApplication10\WebApplication10.csproj.

  NuGet Config files used:
      C:\Users\foo\AppData\Roaming\NuGet\NuGet.Config
      C:\Program Files (x86)\NuGet\Config\Microsoft.VisualStudio.Offline.config

  Feeds used:
      https://api.nuget.org/v3/index.json
      C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\

PS C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10> dotnet build .\WebApplication10.sln
Microsoft (R) Build Engine version 15.1.548.43366
Copyright (C) Microsoft Corporation. All rights reserved.

C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\docker-compose.dcproj : error MSB4019: The imported project "C:\Program Files\dotnet\sdk\1.0.0\Sdks\Microsoft.Docker.Sdk\Sdk\Sdk.props" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.
  WebApplication10 -> C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\WebApplication10\bin\Debug\netcoreapp1.1\WebApplication10.dll

Build FAILED.

C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10\docker-compose.dcproj : error MSB4019: The imported project "C:\Program Files\dotnet\sdk\1.0.0\Sdks\Microsoft.Docker.Sdk\Sdk\Sdk.props" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:00:04.20
PS C:\Users\foo\Documents\Visual Studio 2017\Projects\WebApplication10>

Environment data

dotnet --info output:

.NET Command Line Tools (1.0.1)

Product Information: Version: 1.0.1 Commit SHA-1 hash: 005db40cd1

Runtime Environment: OS Name: Windows OS Version: 10.0.14393 OS Platform: Windows RID: win10-x64 Base Path: C:\Program Files\dotnet\sdk\1.0.1

[livarcocc]

cc @MichaelSimons

This is an SDK that ships only with VS. It is not supported in the CLI.

[billpratt]

Right, completely understand that. By adding the docker-compose project feature to a solution completely breaks running dotnet restore and dotnet build on a solution file. Since the dcproj is a VS2017 feature, I dont expect dotnet to build it, but perhaps ignore it.

The microsoft/aspnetcore-build:1.0-1.1 docker image has "fixed" this by adding the SDK in the dotnet path. Howerver, I hope this is a temporary workaround.

root@8ca61219426a:/usr/share/dotnet# find / -xdev 2>/dev/null -name "Microsoft.Docker.Sdk"
/usr/share/dotnet/sdk/1.0.1/Sdks/Microsoft.Docker.Sdk

[livarcocc]

There is a PR out right now add this SDK to the CLI, which should address this issue.

https://github.com/dotnet/cli/pull/6180

[billpratt]

Great! Thanks for the quick turn around

[jgreene]

It appears this issue was closed since dotnet/cli#6180 is supposed to resolve it. Then dotnet/cli#6180 was closed because it requires the Microsoft.Docker.SDK to be open sourced before it can be merged. So how are users supposed to resolve this issue going forward?

[philcontrolf1]

@jgreene In the short term, you just need to copy the appropriate files from the VS 2017 installation (C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\Sdks\Microsoft.Docker.Sdk) to where dotnet expects to find them (C:\Program Files\dotnet\sdk\1.0.1\Sdks\Microsoft.Docker.Sdk on Windows, /opt/dotnet/sdk/1.0.1/Sdks/Microsoft.Docker.Sdk on *nix). Longer term hopefully the SDK will be open sourced and these files can be distributed...

[billpratt]

@livarcocc I propose that this issue be re-opened since there is no fix currently in place except for manually copying SDK files.

[jgreene]

@philcontrolf1 thank you for the solution, this worked for me

[AlexandreOuellet]

Manually copying the files also worked for me, but I also suggest to reopen this issue, as it is indeed an issue until the files are distributed with the CLI.

[philcontrolf1]

With vague apologies for the self-promotion, having put together a whole pipeline of .NET Core 1.1, Visual Studio 2017 and Docker on TeamCity, I wrote a blog post for work on it - hopefully it will help anyone else trying to do this.

[khayes]

I also ran into this today. Copying the files is non-ideal as we run dotnet restore on a dynamic build system and having to do this on every build agent is awkward.

[philcontrolf1]

@khayes Our build agent bootstrapper copies the files on - see my blog post for details.

[opless]

In the community edition the sdk is: "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\Sdks\Microsoft.Docker.Sdk"

... and for 2.0.0-preview1-005977 on OSX you need to put the files here: /usr/local/share/dotnet/sdk/2.0.0-preview1-005977/Sdks/

[rainersigwald]

Since these extensions are installed to VS's copy of msbuild but not dotnet's, can you run the build with msbuild.exe instead of dotnet build?

[philcontrolf1]

@rainersigwald Not on my Linux CI build agents I can't :-)

[srivatsn]

Reopening this issue since it hasn't been resolved yet. I think not being able to build a solution that contains a dcproj without hacks is a significant blocker.

@dazhao-msft - any updates on https://github.com/dotnet/cli/pull/6180

[billpratt]

@srivatsn thank you for re-opening

[dmitry606]

The fix (copying VS Docker SDK to dotnet sdk) does not seem to work for 2.0.0-preview2. I get this error: C:\Program Files\dotnet\sdk\2.0.0-preview2-006497\Sdks\Microsoft.Docker.Sdk\build\Microsoft.VisualStudio.Docker.Compose.targets(80,45): error MSB4022: The result "" of evaluating the value "$(DockerBuildTasksAssembly)" of the "AssemblyFile" attribute in element is not valid. [C:\repos\WebApplication1\docker-compose.dcproj]

Any solutions for this?

[philcontrolf1]

@dmitry606 Seems to work for me:

PS D:\Source\HelloWorld> dir

Directory: D:\Source\HelloWorld

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----       01/08/2017     09:15                bin
d-----       01/08/2017     09:15                HelloWorld
d-----       01/08/2017     09:15                obj
-a----       01/08/2017     09:15            271 docker-compose.ci.build.yml
-a----       01/08/2017     09:15            723 docker-compose.dcproj
-a----       01/08/2017     09:15             14 docker-compose.override.yml
-a----       01/08/2017     09:15            370 docker-compose.vs.debug.yml
-a----       01/08/2017     09:15            264 docker-compose.vs.release.yml
-a----       01/08/2017     09:15            136 docker-compose.yml
-a----       01/08/2017     09:15           1479 HelloWorld.sln

PS D:\Source\HelloWorld> dotnet --version
2.0.0-preview2-006497
PS D:\Source\HelloWorld> dotnet build .\HelloWorld.sln
Microsoft (R) Build Engine version 15.3.388.41745 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  HelloWorld -> D:\Source\HelloWorld\HelloWorld\bin\Debug\netcoreapp1.1\HelloWorld.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:01.97
PS D:\Source\HelloWorld> dotnet run --project .\HelloWorld\HelloWorld.csproj
Hello World!

Have you got exactly the same SDK files in exactly the same location?

$ sha1sum /cygdrive/c/Program\ Files/dotnet/sdk/2.0.0-preview2-006497/Sdks/Microsoft.Docker.Sdk/Sdk/*
a30e05d5e5d250884a2e5a29f4730f95e0fb6f85 */cygdrive/c/Program Files/dotnet/sdk/2.0.0-preview2-006497/Sdks/Microsoft.Docker.Sdk/Sdk/Sdk.props
cea93148ba2a590fbda25bfba0ed561f510e6c4a */cygdrive/c/Program Files/dotnet/sdk/2.0.0-preview2-006497/Sdks/Microsoft.Docker.Sdk/Sdk/Sdk.targets

(/cygdrive/c is just C:\ in case you're not a Cygwin person)

[dmitry606]

@philcontrolf1 Yes, it's exactly the same location

`c:\repo\HelloWorld>cd C:\Program Files\dotnet\sdk\2.0.0-preview2-006497\Sdks\Microsoft.Docker.Sdk\Sdk

C:\Program Files\dotnet\sdk\2.0.0-preview2-006497\Sdks\Microsoft.Docker.Sdk\Sdk>dir

Directory of C:\Program Files\dotnet\sdk\2.0.0-preview2-006497\Sdks\Microsoft.Docker.Sdk\Sdk

07/31/2017 03:59 PM

. 07/31/2017 03:59 PM .. 07/28/2017 06:50 PM 1,927 Sdk.props 07/28/2017 06:50 PM 1,220 Sdk.targets 2 File(s) 3,147 bytes `

[philcontrolf1]

@dmitry606 You've definitely got different files though - my Sdk.props is 1750 bytes and my Sdk.targets is 1264 bytes. Both copied from C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\Sdks\Microsoft.Docker.Sdk\Sdk with Visual Studio 15.2.26430.16, although I don't think they've changed in a while.

[dmitry606]

@philcontrolf1 Yep.. looks like that. I copied the SDK from VS 2017 Preview 2. Probably they broke the SDK in preview2. For now I decided to use Visual Studio for build, hope it will be fixed soon..

[dazhao-msft]

@dmitry606 It looks like that you copied the Microsoft.Docker.Sdk folder from VS 15.3 Preview, under which there are 3 folders: build, Sdk, tools. Please delete build and tools and only keep Sdk.

[dmitry606]

@dazhao-msft It worked! Thanks!!

[milla]

now Linux .net core 2.0 sdk doesn't support docker? I got docker-compose.dcproj : error MSB4236: The SDK 'Microsoft.Docker.Sdk' specified could not be found.

[philcontrolf1]

@milla Have you copied the appropriate SDK as listed in this thread? If so, exactly which files did you copy, where did you copy them from, and where did you copy them to? A number of us have this working under Preview 2, so it's definitely not completely broken.

[milla]

@philcontrolf1 thanks for checking. It's confirmed that this is a mistake, I built some vsts build agents for different purposes of testing, some of them don't have docker installed. and the agent I used and occurred this problem just doesn't have. Now everything is working perfectly.

[liri2006]

Any ETA on fix?

[mmisztal1980]

This is still broken after .NET Core 2.0 release. Can anyone comment on the state of affairs?

I've tried to do a dotnet build ./MySolution.sln on a solution which contained a project with enabled Docker (linux) support.

dotnet build .\Cluster.sln
D:\Checkout\netcore20ACS\cluster-seed-svc\docker-compose.dcproj : error MSB4236: The SDK 'Microsoft.Docker.Sdk' specified could not be found.
D:\Checkout\netcore20ACS\cluster-seed-svc\docker-compose.dcproj : warning NU1503: Skipping restore for project 'D:\Checkout\netcore20ACS\cluster-seed-svc\docker-compose.dcproj'. The project file may be invalid or missing targets required for restore. [D:\Checkout\netcore20ACS\cluster-seed-svc\Cluster.sln]
Microsoft (R) Build Engine version 15.3.409.57025 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

D:\Checkout\netcore20ACS\cluster-seed-svc\docker-compose.dcproj : error MSB4236: The SDK 'Microsoft.Docker.Sdk' specified could not be found.
  Cluster.InternalApi -> D:\Checkout\netcore20ACS\cluster-seed-svc\src\Cluster.InternalApi\bin\Debug\netcoreapp2.0\Cluster.InternalApi.dll

Build FAILED.

D:\Checkout\netcore20ACS\cluster-seed-svc\docker-compose.dcproj : error MSB4236: The SDK 'Microsoft.Docker.Sdk' specified could not be found.
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:00:04.44

[philcontrolf1]

@mmisztal1980 You still need to copy the files in as detailed here, because Microsoft still haven't open-sourced them. @srivatsn, @dazhao-msft: it would be really good if we could have a progress update here, even if it's just "we're working on it". As you can see, this is hitting a lot of people.

[dasMulli]

While waiting for it to be open sourced was a good idea when the expectation was that it would be open sourced in a few weeks time, it is now becoming increasingly cumbersome. Even mono added the stub SDK to its msbuild distribution https://github.com/mono/msbuild/tree/xplat-master/sdks/Microsoft.Docker.Sdk

[philcontrolf1]

@dasMulli Interesting, they are the official files. Not quite sure how they're ending up in the MIT licensed Mono with a line in them saying "Copyright (C) Microsoft Corporation. All rights reserved." though...

[billpratt]

@livarcocc Can we please have an update on this issue?

[davidmceleney]

A workaround to this - if you're using teamcity, change your build script to execute the commands in your docker-compose file...

Replace: docker-compose docker-compose docker-compose.ci.build.yml up

With: dotnet restore mysolution.sln dotnet publish mysolution.sln -c Release -o ./obj/Docker/publish

removes the dependency on Microsoft.Docker.Sdk...

hope it helps someone!

[matthewDDennis]

I found a work around. Set the build dependencies on the docker-compose project to include ALL other projects. This will allow the other projects to build/publish before dotnet attempts to build/publish the docker project. You still get an error at the end, but it does not abort the build/publish until it doesn't matter.

[felschr]

Wow, this is really annoying. Didn't this work in .NET Core 1.0/1.1? I'm just trying to run my docker-compose.ci.build.yml after updating to .NET Core 2.0. It was working previously.

[philcontrolf1]

@FelschR No, it's never worked.

[felschr]

@philcontrolf1 Interesting... I'm using VSTS (Hosted Linux Preview) and it definetely works with .NET Core 1.0. Maybe the previous aspnetcore-build docker image shipped with the Microsoft.Docker.Sdk?

[philcontrolf1]

@FelschR Maybe. It certainly doesn't work if you build your images from scratch, just installing the .NET Core SDK on top of an empty Linux image.

[billpratt]

@FelschR

Maybe the previous aspnetcore-build docker image shipped with the Microsoft.Docker.Sdk?

The microsoft/aspnetcore-build:1.0-1.1 docker image "fixed" this by adding the SDK in the dotnet path.

[billpratt]

@livarcocc @dazhao-msft @eerhardt Can we please have an update on this? Even a "we are working on it.." would be helpful. More and more people are coming to this issue as well as the closed PR that would have fixed it.

[felschr]

For all those trying to get the docker image (microsoft/aspnetcore-build) working see: https://github.com/aspnet/aspnet-docker/issues/299

Apparently there is also a microsoft/aspnetcore-build:1.0-2.0 image that comes with the Microsoft.Docker.Sdk.

Finally my build is working again thanks to @natemcmaster who provided that solution.

[bruno-david]

Just to summarize the steps I had to perform to make it work on Windows and Linux (Ubuntu):

1. Copy the Microsoft.Docker.Sdk folder from C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\Sdks (Only the Sdk subfolder. Do not copy build and tools subfolders).
2. Then paste it on C:\Program Files\dotnet\sdk\2.0.0\Sdks on Windows or /usr/share/dotnet/sdk/2.0.0./Sdks on Linux (Ubuntu) Then dotnet build SolutionName.sln will work fine.

These steps will fix both errors:

• error MSB4236: The SDK 'Microsoft.Docker.Sdk' specified could not be found.
• error MSB4022: The result "" of evaluating the value "$(DockerBuildTasksAssembly)" of the "AssemblyFile" attribute in element is not valid.

[mkoertgen]

Ok, the suggested workaround fixes dotnet restore and dotnet build.

How about dotnet run?

Unable to run your project.
Please ensure you have a runnable project type and ensure 'dotnet run' supports this project.
A runnable project should target a runnable TFM (for instance, netcoreapp2.0) and have OutputType 'Exe'.
The current OutputType is 'DockerCompose'.

Seems that the output type is not supported. Any idea on how to work around this, too?

[mlowijs]

As another workaround, one could run dotnet sln MySolution.sln remove docker-compose.dcproj to remove the project reference.

This is what I do in my VSTS build definition.

[Vlaaaaaaad]

Will this ever be fixed?

[sheigel]

After so many years of generating frustration it seems that Visual Studio still takes priority to having a working/proper CLI. It seems that having a strong CLI that always works is much more useful than having a slow running IDE that either way should not be installed on the CI machine. But for the sake of having this single IDE more click-friendly you are willing to break something so foundational as platform independence.

I was looking through the eShopOnContainers. I was so happy to have dotnet, docker and Mac working together. I wanted to start with the WebMonolithic solution. But I cannot even dotnet restore.

You've put a lot of effort in making dotnet core a good platform for the current times. Is it a bad idea to make sure that whatever feature is added it is working first on the CLI and then see what can be done for VS?

[billpratt]

@DamianEdwards @shanselman Sorry to include you two on this issue but we haven't received feedback on this issue for some time now and we are looking for help. More and more people are coming to this issue as well as the closed PR that would have fixed it.

[tyronegroves]

So no fix?

[shanselman]

ping @richlander ?