MethodsAndPractices / vsteam

PowerShell module for accessing Azure DevOps Services and Azure DevOps Server (formerly VSTS or TFS)
https://methodsandpractices.github.io/vsteam-docs/
MIT License
445 stars 155 forks source link

Run Builds in Docker Container Locally #328

Closed SebastianSchuetze closed 4 years ago

SebastianSchuetze commented 4 years ago

I have been playing around to run the builds with Pester 5 in container locally for the following configurations

It uses docker and I have created 2 images with our modules preinstalled. There were some problems to run PS5.1, but I managed to have this run. I think it still needs some minor changes, but 95% I feel it is good enough to have it for developers to use.

They all seem to work. IN between build the daemon switches from windows to linux since linux based containers cannot run with windows containers in parallel.

Tell me what you think and if it works for you @DarqueWarrior

DarqueWarrior commented 4 years ago

$PSVersionTable.Platform on my machine (Windows 10, PowerShell 5.1) returns nothing because it was not supported then. and causes the following errors:

WARNING: Platform is not Win32NT based but ''. Windows container do not work on linux based systems. Ignoring windows containers...
error during connect: Get http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.40/images/json?filters=%7B%22reference%22%3A%7B%22vsteam%2Flinux%22%3Atrue%7D%7D: open //./pipe/docker_engine: The system cannot find the file specified. In the default daemon configuration on Windows, the docker client must be run elevated to connect. This error may also indicate that the docker daemon is not running.
unable to prepare context: unable to evaluate symlinks in Dockerfile path: CreateFile R:\repos\vsteam\build\docker: The system cannot find the file specified.
error during connect: Get http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.40/containers/json?all=1&filters=%7B%22name%22%3A%7B%22vsteam_linux_ps7_tests%22%3Atrue%7D%7D: open //./pipe/docker_engine: The system cannot find the file specified. In the default daemon configuration on Windows, the docker client must be run elevated to connect. This error may also indicate that the docker daemon is not running.
error during connect: Get http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.40/containers/json?filters=%7B%22name%22%3A%7B%22vsteam_linux_ps7_tests%22%3Atrue%7D%7D: open //./pipe/docker_engine: The system cannot find the file specified. In the default daemon configuration on Windows, the docker client must be run elevated to connect. This error may also indicate that the docker daemon is not running.
C:\Program Files\Docker\Docker\resources\bin\docker.exe: error during connect: Post http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.40/containers/create?name=vsteam_linux_ps7_tests: open //./pipe/docker_engine: The system cannot find the file specified. In the default daemon configuration on Windows, the docker client must be run elevated to connect. This error may also indicate that the docker daemon is not running.
See 'C:\Program Files\Docker\Docker\resources\bin\docker.exe run --help'.
error during connect: Get http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.40/containers/vsteam_linux_ps7_tests/json: open //./pipe/docker_engine: The system cannot find the file specified. In the default daemon configuration on Windows, the docker client must be run elevated to connect. This error may also indicate that the docker daemon is not running.
Exception calling "Substring" with "2" argument(s): "StartIndex cannot be less than zero.
Parameter name: startIndex"
At R:\repos\vsteam\tools\docker\Run-ContainerTests.ps1:157 char:10
+          $versiontable = $output.Substring($outputFirst, $ouputLast+1 ...
+          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : ArgumentOutOfRangeException

error during connect: Post http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.40/containers/vsteam_linux_ps7_tests/wait?condition=: open //./pipe/docker_engine: The system cannot find the file specified. In the default daemon configuration on Windows, the docker client must be run elevated to connect. This error may also indicate that the docker daemon is not running.

Get-OperatingSystem works everywhere and is a better way to check OS.

I changed it locally but still getting different errors. I will keep playing with it.

DarqueWarrior commented 4 years ago

I tried to push my change but got an error

Permission to razorspoint/vsteam.git denied to Darquewarrior.
DarqueWarrior commented 4 years ago

I am getting close. I have the Linux version working on Windows. But the Windows images are failing to build.

DarqueWarrior commented 4 years ago

I can't seen to get my Windows containers access to the internet so all the module installs are failing. Did you have to do anything special to get your Window containers to see the internet?

DarqueWarrior commented 4 years ago

I got internet in my Windows VMS but following this https://github.com/docker/for-win/issues/2760

DarqueWarrior commented 4 years ago

Now that I have it working this is pretty cool! I wonder if we should just add a -UseDocker switch to the build-module.ps1 and roll this into that script.

SebastianSchuetze commented 4 years ago

It is strange. I don't see the PR checkbox to give you write access. I just gave you maintainer access to my fork repo!

I hope you can commit now to it.

SebastianSchuetze commented 4 years ago

About integrating it into the build script. I was thinking to refactor some of the build files we use into an extra folder!

I would suggest something like "tools/XYZ". There we put an extra structure for everything.

DarqueWarrior commented 4 years ago

We can discuss during our meeting. I will try to have it this week.

DarqueWarrior commented 4 years ago

Testing on my Linux machine now. Looks to be working. I will move to Mac to test next.

DarqueWarrior commented 4 years ago

Still having issues pushing my changes to this PR. So I will create a new one with my changes. I have tested it on all three platforms. This is a really cool idea! Thanks.

DarqueWarrior commented 4 years ago

Merged in a different PR.