search

linuxkit / lcow

Linux containers on Windows built with LinuxKit
Apache License 2.0
225 stars 42 forks source link

File ULIMITs for Linux containers #34

Open dwrightpulse opened 5 years ago

dwrightpulse commented 5 years ago

Hi,

I'm working on a PoC build to run Linux and Windows-based containers side by side on WS2019. To enable the side by side nature I have deployed the WS2019 host as per the instructions at https://github.com/linuxkit/lcow, using dockerd.exe and docker.exe from the master project from 13/02/19 and the v4.14.35-v0.3.9 release for the initrd.img and kernel files.

I can successfully run Linux containers, but when running the influxdb:1.7.4 container from https://hub.docker.com/_/influxdb, and writing to the database the console output report that it is unable to write data due to too many open files in use. We use this image in Azure ACI and have no issues with it.

Having done some investigation, the ULIMIT for files in the running container when running in LCOW on the 2019 host are 1024 and when running in ACI or Docker Desktop on Win 10-1809, the limits are over 1 million (see attached screenshots)

ulimit influx in azure influx ulimit in docker desktop ulimit influx in docker lcow on ws2019

I'm posting here as concluding this is an issue with the LinuxKit kernel that this solution uses. I have tried to see if I can use alternate kernel files to confirm if this is the issue (https://docs.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/linux-containers suggested Kata and Clear Linux kernels (but I have got nowhere with these 2 options so far)) and I am unable to over-ride the default files ulimit in the daemon.json file (as per https://docs.docker.com/engine/reference/commandline/dockerd/) as when starting the dockerd daemon it reports that this is not configurable in Windows (see further attached screenshot)

default-ulimit in docker lcow on ws2019

As a way of progressing, I'm happy to try the Kata or Clear kernel option if anyone can give me some guidance on how to get hold of the right files and configure LCOW to use them....

justincormack commented 5 years ago

That limit will need to be set in https://github.com/Microsoft/opengcs/blob/master/init/init.c so you may want to open an issue there.

dwrightpulse commented 5 years ago

Thanks Justin, I'll do that...

jstarks commented 5 years ago

We can increase the default there, but we should also fix Docker to support —ulimit and —default-ulimit when LCOW is supported. @jhowardmsft

dwrightpulse commented 5 years ago

@jhowardmsft @jstarks - Any update on this please?

lowenna commented 5 years ago

Working on higher priority stuff right now. It’s on the list

dwrightpulse commented 5 years ago

Thanks John, any idea on timescale. My PoC timescale is end of May, is that feasible?

lowenna commented 5 years ago

I can’t provide any timescale.

dwrightpulse commented 5 years ago

OK, can you give any input on using alternate kernels within LCOW to see if this circumvents the issue?

lowenna commented 5 years ago

Not really. I was going to look at fixing it from moby not via an alt kernel

dwrightpulse commented 5 years ago

@jhowardmsft - are you any closer to fixing this in Moby?

Will the Build announcements ref WSL2 have any effect on how this is implemented in the future?