Windows Containers Support

[Silvenga]

I'm trying to prototype using crank in DnD (Docker-in-docker) so I'm abusing the the docker hostname argument on the agent and the arguments job property.

That said, all these problems should occur outside of the container as well:

Windows containers docker commands are generated by: https://github.com/dotnet/crank/blob/ea81a866146a5733eba9aac74c79afd66cb8efd6/src/Microsoft.Crank.Agent/Startup.cs#L1911

Where hostname is the value of the OS's hostname or the value passed form --docker-hostname. That means the --ip is set to a hostname, which is invalid (must be an IP).

[STDERR] docker: Error response from daemon: invalid IPv4 address: 13ad8aac2fae

If --docker-hostname is set to an IP e.g. --docker-hostname 172.21.154.177, the run command fails further with:

[STDERR] docker: Error response from daemon: network SELF not found.

I'm unsure where the SELF network comes from, I don't think it's a built in network.

---

Completely unrelated, true DnD support would be amazing, would this project be open to supporting that? I can make the required changes (resolve an IP from the constructed container and use that in-place of the docker hostname, could be an opt-in flag).

[sebastienros]

The SELF network was a way to get host networking working with hyper-v last time I tried it on Windows. Haven't followed the improvements for containers on Windows since then, and it is ok to change it if necessary.

Not sure what DnD means, but on Linux I am already running Crank-Agent inside a Docker container, and it successfully create containers. Check the file I am using on Linux: https://github.com/dotnet/crank/blob/main/docker/agent/Dockerfile It's using host network on Linux, which might be why it's so easy.

[Silvenga]

I'm not sure there is a current way to execute Windows containers in host network mode (at least not documented). I'm using Docker Desktop for prototyping, but it would eventually execute under Docker EE. There could definitely be differences there.

Docker in Docker would look like the crank-agent running inside a container and building new containers that run side-by-side. Crank would then reach out to it's sibling container to execute the benchmark.

This would prevent having to build custom images for each application under benchmark.

My working prototype looks like this:

docker build --tag crank-agent .

try {
    Write-Host "Creating network."
    docker network create -d "nat" --subnet "10.244.0.0/24" SELF

    Write-Host "Starting container."
    $agentContainer = docker run -d --rm -p 5010:5010 -v \\.\pipe\docker_engine:\\.\pipe\docker_engine --network SELF --ip 10.244.0.2 --isolation process crank-agent crank-agent --docker-hostname 10.244.0.3
    Write-Host "Attaching to stdout, Ctrl + C to exit."
    docker logs -f $agentContainer
}
finally {
    Write-Host "Stopping container."
    docker stop $agentContainer

    $(docker ps --filter label=benchmarks -q) | ForEach-Object {
        Write-Host "Removing spawned container $_."
        docker rm --force $_
    }

    Write-Host "Removing network."
    docker network rm SELF
}

Note I'm exposing dockerd inside the container with 10.244.0.3 being the IP of the container that crank spawns (yeah, abusing the docker-hostname). This allows Docker in Docker where host networking doesn't exist.

---

Background - I'm prototyping out a benchmarking solution that works for both .NET Framework (IIS running in containers) and .NET Core/5 (Linux and Windows containers). Our CI's (Azure DevOps) servers has no tooling installed, all builds must use Docker images to pull in build dependencies - which is why I'm trying to work with DnD.

---

All that said, I'm starting to see limitations of this approach - collection performance counters has problems which cases nullrefs - so definitely not a well traveled path.

How is performance counter collection expected to work? It looks like perfview is running on the agent (so not in the same container as the app). Is it relying on the loose sandboxing under Linux of containers to collect data? (Messing with running Windows containers in process mode isolation to compensate)

[sebastienros]

I'm not sure there is a current way to execute Windows containers in host network mode

We stopped benchmarking in Docker on Windows since installing Hyper-V was impacting all benchmarks by 20% on the network stack (virtual switches). So we only benchmark locally deployed apps on Windows now.

Docker in Docker would look like the crank-agent running inside a container and building new containers that run side-by-side. Crank would then reach out to it's sibling container to execute the benchmark.

This is what I am doing on Linux.

Our CI's (Azure DevOps) servers has no tooling installed

I am also using AzDo to drive the benchmarks, but only to trigger them remotely, not to build anything. That said crank supports "local folders" to be uploaded to the agent, and also binaries. So you could build everything on AzDo, and point to the build folder. Crank will then zip the content, and run the binary on the crank agent. If the two are not in the same network, then it's not yet supported, but that's one of my priorities, by being able to upload the assets on a blob storage.

Perfview is run from the Agent, so not sure about Docker compatibility. On Linux we use perf/perfcollect and it works fine. Dotnet-Monitor seems to work as a side-car, so maybe it will also work on Windows. Sincecrank support dotnet-trace, there might be some things that are common.

[Silvenga]

Dotnet-Monitor seems to work as a side-car, so maybe it will also work on Windows

Ah, unfortunately I need to support Framework for as long as Microsoft supports it (performance testing for an APM). If Microsoft could deprecate .NET 3.5 that would be great. 😸

I guess the main difference between what you're doing in Linux vs Docker in Docker is that DnD doesn't care about where things are e.g. the two containers could be executing on different hosts. This means Docker abstraction should be used e.g. Docker networking. But this is 1) limiting e.g. no volume mounts 2) requires application awareness of DnD e.g. getting other containers IP addresses.

So it sound like:

• Windows containers don't work right now since the SELF network changed at some point. This breaks host networking.
• Windows containers can't really operate in Docker in Docker mode since the agent assumes that all containers are on the same network stack.

It doesn't seem like Windows containers are really a good long term feature for this project (due to the current assumptions) - so even if I were to make a pr to add support it would at best be brittle, or worse, limit other features (although I'm totally willing if otherwise).

---

I am also using AzDo to drive the benchmarks, but only to trigger them remotely, not to build anything.

I think that's where the strength of cranks comes in - decoupling the different components adds so much flexibility! Unfortunately I'm somewhat limited to running on a pool of existing transient Azure VM's (so definitely not a round peg).

Lot's of really cool possibilities of crank!