Add better Windows CNB contribution workflow documentation

micahyoung commented 4 years ago

Description

Contributing Windows features, or debugging unintentional Windows CI failures, is complex and has a steeper learning curve that other OSes, especially for Windows containers.

Proposed solution

Document the process of Windows development and debugging by setting up a minimal Windows dev environment. This should assume the following:

A minimally interactive script to create a new, dedicated machine to be used by the developer
- Prompts for credentials to set on the new machine
Creates a Virtual Machine on a cloud provider or locally, with:
- Windows 10 Enterprise 1809 [1]
- Docker for Windows installed
- Windows Subsystem for Linux (latest Ubuntu) installed [2]
- SSH Server running (ssh-agent forwarding should work to allow SSH cloning)
- Remote Desktop access enabled
- All build tools required for a successful make with pack
- A valid Windows license, when on a cloud provider
- Up-to-date Windows updates

Describe alternatives you've considered

Creating VMs with cloud provider GUIs would still require lots of manually installation steps
Pre-built vagrant images, but they would not be properly licensed and require regular maintenance for Windows updates
Sharing a single, pre-built machine isn't viable for community contributors

Additional context

Stefan Scherer already has a great packer implementation that could ported to use additional builders like googlecompute, azure-arm or others.
[x] This feature should be documented somewhere

Notes

[1] 1809 will be the most common OS version for k8s and corporate customers [2] Microsoft implies this can only be done by the Windows store but it's much easier using Ubuntu's appx

jromero commented 4 years ago

@micahyoung do we have an owner for this issue predetermined?

micahyoung commented 4 years ago

I'm happy to take a first pass. I'll at least come up with something other folks can try out and refine or help us make another attempt. Sound ok?

micahyoung commented 4 years ago

Here's a minimal WIP approach so far.

https://github.com/buildpacks/pack/wiki/Windows-WCOW-Docker-Daemon-on-Cloud-VM

aemengo commented 3 years ago

Thoughts:

$dockerConfig.hosts = @("tcp://127.0.0.1:2375", "npipe://")

Exposing the docker daemon on tcp feels like a dip in security to me. Especially if the user is only expected to work via an ssh session.

$dockerConfig."insecure-registries" = @("0.0.0.0/0")

While I'm not uneasy about this from a security standpoint, locking down the IP range bodes better with the principle of least privilege.

$IPAddress=(Get-NetIPAddress -InterfaceAlias ((Get-NetRoute "0.0.0.0/0").InterfaceAlias) -AddressFamily IPv4)[0].IPAddress

We would want to make sure that the private IP is what is making it into the host file. The last time I tried a public IP, the acceptance tests were routinely failing due a timeout. Mileage may vary with the IAAS though.

go get -u github.com/kyoh86/richgo

I think we should be setting up richgo on all the machines were we expect the tests to run. The output is much easier to parse.

micahyoung commented 3 years ago

Thoughts:

$dockerConfig.hosts = @("tcp://127.0.0.1:2375", "npipe://")

Exposing the docker daemon on tcp feels like a dip in security to me. Especially if the user is only expected to work via an ssh session.

Good call. One alternative I've been considering is making a container image with a small binary that proxies TCP traffic to the socket, instead of the daemon opening the port itself. Then, it's only active when the containers is explicitly started (from RDP or whatever), and we could potentially add better security mechanisms (like TLS, IP whitelisting, auto shutoff or something). But that might not be needed, if we chose to not support DOCKER_HOST=... make with this setup.

$dockerConfig."insecure-registries" = @("0.0.0.0/0")

While I'm not uneasy about this from a security standpoint, locking down the IP range bodes better with the principle of least privilege.

$IPAddress=(Get-NetIPAddress -InterfaceAlias ((Get-NetRoute "0.0.0.0/0").InterfaceAlias) -AddressFamily IPv4)[0].IPAddress

We would want to make sure that the private IP is what is making it into the host file. The last time I tried a public IP, the acceptance tests were routinely failing due a timeout. Mileage may vary with the IAAS though.

Yeah, I think we'll have to use the IP host.docker.internal if it's set, due to pack's current test registry implementation. For Window Server, we do define host.docker.internal ourselves, so I think it could be potentially be the IP address of vEthernet (nat) which I believe is consistently named. For Windows 10, the value of host.docker.internal is a public IP but should still work and wouldn't be covered by these docs anyways.

Supporting DOCKER_HOST=... make also shouldn't be supported in this setup, as you mentioned, the tests will time out for non-local Windows daemons. In fact, it might be better to invest some work in allowing pack to do export DOCKER_HOST=... ; docker build ....; docker run -v <socket> ... make to run the tests inside a Windows container with a bind-mounted socket (much like lifecycle's make docker-run-windows). This would make the tests run as WCOW (like CI), have a consistent way to copy code the daemon (docker build), avoid timeouts since at runtime everything is on the daemon, and just use host.docker.internal's IP for everything.

go get -u github.com/kyoh86/richgo

I think we should be setting up richgo on all the machines were we expect the tests to run. The output is much easier to parse.

👍

buildpacks / pack