search

containers / podman

Podman: A tool for managing OCI containers and pods.
https://podman.io
Apache License 2.0
22.41k stars 2.31k forks source link

Support running Windows Containers on linux #8136

Open seanyoung opened 3 years ago

seanyoung commented 3 years ago

I realise this might be out of scope but I thought I'd raise it so it can be considered.

Windows Containers are slightly modified docker containers, which contain windows executables and the windows registry. These file systems could be mounted with case-insensitive options, and then a very thin wine shim could run the actual binaries. From what I can see, all the necessary executables and dlls are included in the images (e.g. cmd.exe).

https://docs.microsoft.com/en-us/virtualization/windowscontainers/

This would be super-useful to be able to run e.g. windows builds on linux, see and example of a windows dockerfile here: https://github.com/hyperledger/solang/blob/792df33e712f1142da515b7b69f4a4fe7965b5e4/build/build-llvm-windows.dockerfile

rhatdan commented 3 years ago

This would definitely would require community to work on. Not something the core developers are likely to work on.

github-actions[bot] commented 3 years ago

A friendly reminder that this issue had no activity for 30 days.

trevor-viljoen commented 3 years ago

As much as I don't want to deal with Windows Containers, we have many Windows application workloads in the pipeline that we need to containerize. I would like to use Quay's builders to facilitate some of this work. However, until podman/buildah/libpod support this it won't be possible.

github-actions[bot] commented 3 years ago

A friendly reminder that this issue had no activity for 30 days.

rhatdan commented 3 years ago

No progress and no volunteers.

github-actions[bot] commented 3 years ago

A friendly reminder that this issue had no activity for 30 days.

vrothberg commented 3 years ago

I am closing this issue in favor of https://github.com/containers/podman/issues/8452

seanyoung commented 3 years ago

8452 is about running podman on windows/mac. This issue about running windows containers on podman (on linux). Not the same issue.

vrothberg commented 3 years ago

Thanks, @seanyoung! I missed that you want to run Windows containers on Linux.

github-actions[bot] commented 3 years ago

A friendly reminder that this issue had no activity for 30 days.

rhatdan commented 3 years ago

This would require something like kata to run qemu within a container.

seanyoung commented 3 years ago

@rhatdan no this would not require qemu. It would require a wine. I imagine there would be base layer with wine, and then the windows image as /c/. Then wine runs /c/windows/system32/cmd.exe (or whatever is specified as the startup executable.

The wine can be stripped down version as no gui components are required. Windows doesn't allow gui things to run in windows containers either.

rhatdan commented 3 years ago

@seanyoung crun has the ability to load shared libraries when running containers, we are doing something with this when setting up libkrun. If we had a similar library around wine, we might be able to pull this off. @giuseppe WDYT?

Then we could launch crun with a wrun OCI Link.

@seanyoung Would this be something you would be interested in working on?

seanyoung commented 3 years ago

@rhatdan this is something I'd like to work on, but realistically I don't know when I'll find the time. I'm already working on other open source projects. Not enough time in the day is one of life's greatest disappointments.

Having said that, I had started working on making

$ podman pull mcr.microsoft.com/windows/servercore:ltsc2019
Trying to pull mcr.microsoft.com/windows/servercore:ltsc2019...
  no image found in manifest list for architecture amd64, variant "", OS linux
Error: Error choosing an image from manifest list docker://mcr.microsoft.com/windows/servercore:ltsc2019: no image found in manifest list for architecture amd64, variant "", OS linux

work, but this required removing some assumptions across the code. Should pull have a --windows option?

seanyoung commented 3 years ago

Hmm looks like podman pull now has --os and --platform options. It's been a while since I looked at this

rhatdan commented 3 years ago

$ podman pull --os windows mcr.microsoft.com/windows/servercore:ltsc2019 Trying to pull mcr.microsoft.com/windows/servercore:ltsc2019... Getting image source signatures Copying blob 5ff1512f88ec [=>------------------------------------] 36.6MiB / 708.8MiB Copying blob 4612f6d0b889 [--------------------------------------] 1.3MiB / 1.6GiB ...

rhatdan commented 3 years ago

Sadly this failed. Not sure why.

 $ podman pull --os windows mcr.microsoft.com/windows/servercore:ltsc2019
Trying to pull mcr.microsoft.com/windows/servercore:ltsc2019...
Getting image source signatures
Copying blob 5ff1512f88ec [================>---------------------] 316.2MiB / 708.8MiB
Copying blob 5ff1512f88ec done  
Copying blob 4612f6d0b889 done  
Copying config 3eaa9ebbf5 done  
Writing manifest to image destination
Storing signatures
  Error processing tar file(exit status 1): link /Files/Program Files/common files/Microsoft Shared/Ink/en-US/micaut.dll.mui /Files/Program Files (x86)/common files/Microsoft Shared/ink/en-US/micaut.dll.mui: no such file or directory
Error: Error committing the finished image: error adding layer with blob "sha256:5ff1512f88ecd4ff0a91ade34d43733c48834433e35f062f80e6e70d706f8d73": Error processing tar file(exit status 1): link /Files/Program Files/common files/Microsoft Shared/Ink/en-US/micaut.dll.mui /Files/Program Files (x86)/common files/Microsoft Shared/ink/en-US/micaut.dll.mui: no such file or directory
seanyoung commented 3 years ago

Yes, file links seem to work differently. I haven't quite figured out what the difference is.

The image also includes a copy of the windows registry.

Lastly the filesystem is case-insensitive. The linux kernel has support for this, but I'm not quite sure what modifications podman needs to set this up.

rhatdan commented 3 years ago

@rhvgoyal WDYT?

giuseppe commented 3 years ago

@seanyoung crun has the ability to load shared libraries when running containers, we are doing something with this when setting up libkrun. If we had a similar library around wine, we might be able to pull this off. @giuseppe WDYT?

that is something we could add. Is it possible to load wine from a .so so we don't have to bind mount external files into the container? Does it require to load other files (that won't be accessible from the container) in order to work?

giuseppe commented 3 years ago

Lastly the filesystem is case-insensitive. The linux kernel has support for this, but I'm not quite sure what modifications podman needs to set this up.

it may need to setup a different storage on a case insensitive file system (ext4 and xfs should be fine). That will be a problem if we want the linux and windows containers to use the same storage.

seanyoung commented 3 years ago

It might be possible to use libwine.so so we don't have to bind mount external files. I think to get things up a running more quickly, we should first work with a bind mount. Also, since this hasn't been attempted yet, we don't know what hurdles we are going to face.

github-actions[bot] commented 3 years ago

A friendly reminder that this issue had no activity for 30 days.

github-actions[bot] commented 3 years ago

A friendly reminder that this issue had no activity for 30 days.

rhatdan commented 3 years ago

@giuseppe Any chance you could create a spike to see if we could support this?

giuseppe commented 3 years ago

created one: https://issues.redhat.com/browse/RUN-1262

github-actions[bot] commented 2 years ago

A friendly reminder that this issue had no activity for 30 days.

rhatdan commented 2 years ago

I would love to see this move forward.

flouthoc commented 2 years ago

@trevor-viljoen @seanyoung @rhatdan @giuseppe From the issue it seems more like windows-binary-on-linux / wine-inside-container something like this as base image should solve the use-case right ?

Sorry i might have missed it or was not able to infer from discussions in the above thread but could someone help me understand the use-case where runtime explicitly needs to handle wine.


FROM ubuntu:18.04

# Install wget
RUN apt-get update
RUN apt-get install -y wget

# Add 32-bit architecture
RUN dpkg --add-architecture i386
RUN apt-get update

# Install Wine
RUN apt-get install -y software-properties-common gnupg2
RUN wget -nc https://dl.winehq.org/wine-builds/winehq.key
RUN apt-key add winehq.key
RUN apt-add-repository 'deb https://dl.winehq.org/wine-builds/ubuntu/ bionic main'
RUN apt-get install -y --install-recommends winehq-stable winbind

# Turn off Fixme warnings
ENV WINEDEBUG=fixme-all

# Setup a Wine prefix
ENV WINEPREFIX=/root/.net 
ENV WINEARCH=win64
RUN winecfg

# Install Winetricks
RUN apt-get install -y cabextract
RUN wget https://raw.githubusercontent.com/Winetricks/winetricks/master/src/winetricks
RUN chmod +x winetricks
RUN cp winetricks /usr/local/bin

# Install .NET Framework 4.5.2
RUN wineboot -u && winetricks -q dotnet452

# Compile your binary here

# Run Your binary
ENTRYPOINT ["wine", "<your-binary>"]```
seanyoung commented 2 years ago

@flouthoc I think a wine base image is a great way to start this.

There might be advantages to having the runtime explicitly use libwine rather than a wine base image. It might be easier to configure certain wine settings or capture wine logs, if needed. Also using libwine rather than a wine base image feels a lot more robust/native.

Having said all that, presumably using a wine base image will be easier to get this thing working and then later we can consider using libwine.

rhatdan commented 2 years ago

I prefer a ociwine for this. Which is just crun linked against libwine. The bigger problem is the file system issue. Basically Podman or other tools would need a way to have a Windows storage directory which is setup on a different mount point. The windows storage would require the file system to ignore case differences, so that the Wine APPS within the container would work correctly.

github-actions[bot] commented 2 years ago

A friendly reminder that this issue had no activity for 30 days.

rhatdan commented 2 years ago

@flouthoc @giuseppe I would like to setup a meeting to discuss this and figure out if it is possible or close this issue.

carmo-evan commented 2 years ago

I think this becomes even more relevant now with Docker Desktop becoming a paid service.

seanyoung commented 2 years ago

Also docker for linux can't run windows containers. This is a real killer feature

flouthoc commented 2 years ago

Working on a PoC for this. Starting this with bind mount we can switch to linked .so after that.

geoffreysmith commented 2 years ago

@flouthoc I'm following the progress in multiple repos with my own repo (https://github.com/geoffreysmith/boring-euclid) ... seems to be a lot of people saying there's Windows support but not meaning running and building Windows containers. Would be interesting in helping with the development in anyway I can. Is there a discord or something?

giuseppe commented 2 years ago

@flouthoc are you still working on this issue?

@geoffreysmith we can discuss on #podman (libera.chat), the idea is to take advantage of Wine and embed it in crun to be able to run Windows apps. We have not managed yet since Wine uses a lot of shared libraries. The first step is probably to get a static build of Wine

vrothberg commented 2 years ago

Note that the official IRC channel is on https://libera.chat/ now.

flouthoc commented 2 years ago

@geoffreysmith @seanyoung I think wine might not be a good fit for this use case since running static wine in itself is not very well supported officially https://forum.winehq.org/viewtopic.php?t=31388 (someone attempted in this in past) and the amount of shared libraries needed by wine is making plumbing and build extremely difficult for crun.

Also afaik wine has no intention of being used as an embeddable library thus making implementation extremely difficult for such use-cases.

flouthoc commented 2 years ago

I tried removing things from wine but seems after rootfs isolation we also need few dependencies to be part of rootfs, in theory crun could inject those dependencies into rootfs but it seems a base image might be better fit for this.

I might be wrong but could use some hints on this issues. Not sure also i think we misinterpret idea of libwine ( its more for instrumenting windows code rather than using wine as embedded library)

Adding some resources which i used while working on this

giuseppe commented 2 years ago

Note that the official IRC channel is on https://libera.chat/ now.

thanks! Fixed my comment

github-actions[bot] commented 2 years ago

A friendly reminder that this issue had no activity for 30 days.

rhatdan commented 2 years ago

@flouthoc @giuseppe no work lately correct?

flouthoc commented 2 years ago

@rhatdan Doing some research again. Hopefully will update here soon.

github-actions[bot] commented 2 years ago

A friendly reminder that this issue had no activity for 30 days.

github-actions[bot] commented 2 years ago

A friendly reminder that this issue had no activity for 30 days.

flouthoc commented 2 years ago

A variant of this is being implemented for crun so I am removing stale.

seanyoung commented 2 years ago

@flouthoc I guess you mean https://github.com/containers/crun/pull/841 ?

How is that going to help towarrds running windows containers? This is just loading dotnet applications which can already run fine in linux containers.

flouthoc commented 2 years ago

@seanyoung It will be much easier to do similar loading with wine shared libraries as well but again wine has a dependency on huge set of shared libaries. But the idea of above PR is to run executables without runtime layer in container image but will be only scoped to .NET.

Edit: I am hoping a similar bind mount for all the shared libaries for wine should make it work just like we are doing for mono. Since static building of wine is not at all easy. https://forum.winehq.org/viewtopic.php?t=31388