search

rust-lang / docker-rust

The official Docker images for Rust
441 stars 90 forks source link

Windows Support #30

Open fkorotkov opened 6 years ago

fkorotkov commented 6 years ago

Are there any plans to get official Windows Server images? Preferably based of microsoft/windowsservercore image?

sfackler commented 6 years ago

We'd definitely like to have them - the only blocker is figuring out how to make them actually work. There's an attempt here for example: https://github.com/alexcrichton/docker-rust/commit/494237c6d6867cd696fa1f3c4bc1ec31c5f85d0d

nelsonjchen commented 5 years ago

Is there a list of unresolved issues with that attempt?

nelsonjchen commented 5 years ago

🤷‍♂️

I guess a big one is that the build tools are kinda launched in a forked process on that branch and don't specify --wait, causing a premature killing of the intermediate build step and no VS stuff being installed. I'm saying this in case anyone wants to go ahead of me and bang on it. This was something I noticed by looking at examples of Windows Containers in GitHub installing the VS Build Tools.

I have a branch in my fork that definitely hacky, still doesn't work, and still a WIP. Need to rebase and it's kinda annoying that generated "artifacts" are in this repo.

nelsonjchen commented 5 years ago

My branch works, I can do that nifty "Compile your app inside the Docker container" but with Windows thing with the whole --rm thing at the bottom of the README but I don't have it backported to the generate.sh stuff yet. I think I'll go check out how the python Docker image works since they have a windows target there and see what they do.

sfackler commented 5 years ago

Awesome!

I cargo-culted a bunch of the setup here from the python and golang images, so they'd be good places to look for Windows setup as well.

nelsonjchen commented 5 years ago

By the way, if anyone is developing this on Windows 10 and not Windows Server:

If you're on 10 and 1809, replace the docker.exe and dockerd.exe executables from the latest builds of docker from the tip. Microsoft recently sent in a PR and licensing approval to remove Docker's artificial disablement of --isolation=process on Windows client SKUs from the daemon. It's not in Docker Edge yet as of this comment. This means faster development without the Hyper-V VM and visibility with Procmon and so on. Only supported for dev/test.

nelsonjchen commented 5 years ago

I've decided to drop nanoserver from my efforts. It'll probably be great for those multi-stage builds where someone compiles something with Rust from the bigger windowsservercore-based images and copies it to nanoserver for running/runtime but it's probably inappropriate for a build tool. I mean, do you see an alpine image for rust? Though, I guess there's debian slim and a hypothetical alpine version would kinda be oddly supported in a weird way. But this is Microsoft country, and it takes years for them to trim more fat.

sfackler commented 5 years ago

I mean, do you see an alpine image for rust?

The only reason there's no alpine image is that there isn't an x86_64-unknown-linux-musl rustc build yet. #10

nelsonjchen commented 5 years ago

Oh! I didn't see that.

nelsonjchen commented 5 years ago

There are some straggling issues left I guess.

nelsonjchen commented 5 years ago

The GNU variant was predictably crazy easy to implement and definitely not encumbered.

nelsonjchen commented 5 years ago

I have to take a break from this for now. If someone wants to take a crack at getting this upstreamed, have at it. There are MSVC and GNU Dockerfiles in here:

https://github.com/nelsonjchen/docker-rust/tree/windows

I am not sure how to get this integrated in with that generate-stackbrew-library.sh. What's the play here? Run that in WSL with Docker for Windows configured in a mode that supports both Linux and Windows containers exposed to the Docker client inside WSL? I just don't know.

sfackler commented 5 years ago

That just generates the metadata blob that goes over to the docker-library/official-images repo - it doesn't need to run in the same OS as the docker image or anything.

nelsonjchen commented 5 years ago

Oops, I meant update.sh, it depends on something called bashbrew and I'm not familiar with that yet.

sfackler commented 5 years ago

Similarly, update.sh can create a Dockerfile for a Windows image without being run on Windows.

Bashbrew is here: https://github.com/docker-library/official-images/tree/master/bashbrew

nelsonjchen commented 5 years ago

Bashbrew explodes now in WSL with some nil dereference error. Not sure why. Anyway, update.sh still seems to work. I've updated it to handle some new axes.

Along with the msvc and gnu axes. This makes 8 combinations.

Unfortunately, only the ltsc2016 targets build on Appveyor:

https://ci.appveyor.com/project/nelsonjchen/docker-rust/builds/20890075

For Windows containers where the kernel doesn't match the image, nested virtualization is required. These jobs fail to build because Appveyor does not have nested virtualization. Unlike Linux containers, where there's a Linus to scream "WE DO NOT BREAK USERSPACE! Seriously. How hard is this rule to understand? We particularly don't break user space with TOTAL CRAP.", this is not the case for Windows. The userspace is designed to match the kernel.

When the images match, Docker will run the container with process isolation much like on Linux systems. When they don't, Docker will launch a "nano" sized boot of a cut-down Windows VM with a matching kernel version. There's actually about a small but noticeable delay when Docker does this. On client Windows systems, this is the norm and is enforced since client Windows kernels can vary a lot. Only recently in a currently unreleased Docker version did some Microsoftie take off this limitation in a PR. Windows 10 Client 1809 and up can run images matching the kernel with process isolation, though it's branded as for dev and test only. While not currently available and somewhat useless, I added the 1709 and 1803 variants as a small test of building images for users who are running client Windows versions that don't happen to have a LTSC server match, like right now with whatever will come after 1809 and post Windows Server 2019.

Another issue is that I think Appveyor only provides building one job at a time. I could have sworn they used to provide more concurrency in the past for OSS projects. 🤷‍♂️ I might have been confusing it with some other CI service . Right now, these images take about 16 minutes for the GNU variant and 32 minutes for the MSVC variant. Pretend the nested virtualization wasn't an issue. To check all these, it would take about 192 minutes on Appveyor. This list will grow a bit as more 10-year supported LTSC releases of Windows happen.

I have a better, cheaper, and faster proposal. Could we use Azure Pipelines? They provide 10 concurrent Windows (or any OS including Mac or Linux) VMs for free for OSS projects. They currently do not support nested virtualization but this pull request from a Microsoftie is inside this repo they use for generating the images they are using for their service is switching the instance type they are using for building to something that supports nested virtualization. It's un-merged but it's safe to say they're thinking of introducing support for it. Barring that, Azure Pipelines is most likely to have available the in-between non-LTSC builds of Windows such as 1803 right now. Using their service to shotgun build and test all these windows images at once would be great.

nelsonjchen commented 5 years ago

We might have more combinations in the future as well. Like #14 which could double or triple the amount of images.

sfackler commented 5 years ago

I don't have strong preferences on the CI setup - but presumably the other official image repos that have Windows images should have something to base the work here off of?

nelsonjchen commented 5 years ago

Definitely, there's some things to learn from them.

Using the go and python Windows images as reference, I've found the following.

In the meantime, I've also discovered that simply rebasing some of the GNU images atop of nanoserver images simply do not work. nanoserver:sac2016 fails because the powershell module to set TLS 1.2 compatibility simply doesn't exist. Later nanoserver images explode with failures due to some system calls that I guess rustup makes that simply don't work. Cool, nanoserver is definitely a no-go.

So here's my TODO list:

Maybe after all this it might be PR ready. We'll see.

nelsonjchen commented 5 years ago

Alright! I got it refactored to that scheme.

I got a hardcoded azure-pipelines.yml. I'll need to make update.sh generate parts of it or some file it references for templating. I'm a little torn on making a template to be included or do that awk stuff.

https://dev.azure.com/nelsonjchen/docker-rust/_build/results?buildId=15

In the meantime, while hardcoding it, I think there's some issue with the windowsservercore_ltsc2016_msvc image. This isn't related to Rust and it's more likely to be something related to the process isolation mode, ltsc2016, azure, and the VS build tools. This builds fine in hyper-v isolation on my Windows 10 machine. Meanwhile, windowsservercore_ltsc2016_gnu, windowsservercore_1803_gnu, and windowsservercore_1803_msvc run just fine 😮. I'll go read some documentation and posts and see if windowsservercore_ltsc2016_msvc+process isolation + azure just isn't meant to be or something. It works on Appveyor.

gilescope commented 4 years ago

Just a quick check to see if there's been any movement on this front as time has gone by.

IceCodeNew commented 3 years ago

Have we made any progress on this issue these years?

gilescope commented 3 years ago

I think the main issue is no one seems to use windows containers. If they did there might be more movement, but the use case is lacking at the moment.

yodaldevoid commented 3 years ago

As someone who uses Windows containers, there's dozens of us! Dozens!

Jokes out of the way, for my job I am currently creating Windows containers for Rust. I would rather not have to make them myself, but seeing how long this issue has been around and how long the beta issue has been around, I don't think that will be changing any time soon.

sfackler commented 3 years ago

I mean, the only thing blocking this issue is someone getting a Dockerfile that works. If you have those, then feel free to open a PR.

yodaldevoid commented 3 years ago

I was under the impression that there was more needed than just some dockerfiles, but if that is truly all that's needed I'll submit a PR ASAP.

lippertmarkus commented 3 years ago

@yodaldevoid I see you made some progress in https://github.com/yodaldevoid/rust-windows. Can we expect a PR from you?

yodaldevoid commented 3 years ago

Welp, it seems ASAP was two months. Sorry about that, world's been a bit crazy.

Anywho, I've opened #71 with what I've got so far. I'll probably tomorrow to throw together an Azure, Appveyor, or GitHub Actions pipeline from other's previous work. I don't really have any personal stake in getting the GNU toolchains working, but maybe I'll get inspired.

lippertmarkus commented 3 years ago

Thanks a lot!

lippertmarkus commented 2 years ago

For anyone coming here who just wants Windows support to be able to use a compile stage in their Dockerfile for creating a Windows container image:

You can also cross-build a Windows container image on Linux for a Rust application: Example Dockerfile: https://github.com/lippertmarkus/cross-building-windows-and-linux-multi-arch-images/blob/main/windows-examples/rust/Dockerfile How to build with buildkit: https://lippertmarkus.com/2021/11/30/win-multiarch-img-lin/#tldr