search

plasma-mobile / xbuilder

Changeroot Environment for local package builds
0 stars 0 forks source link

set docker ulimit #2

Open hsitter opened 9 years ago

hsitter commented 9 years ago

random thing I remembered:

you must force the open fd limit to be 1024 in docker. otherwise apt and related helpers (such as apt-extracttemplates) IO is going to slow down installation of new packages by an enormous amount: https://github.com/blue-systems/pangea-tooling/blob/master/lib/ci/container.rb#L86

in particular apt-extracttemplates, which should be installed after initial image creation, is turning into a slow pile of crap.

without ulimit installing the entire package tree in kubuntu ci takes almost 2 hours: http://kci.pangea.pub/job/mgmt_daily_promotion_vivid_unstable/531/ with ulimit it takes some 30 minutes: http://kci.pangea.pub/job/mgmt_daily_promotion_vivid_unstable/584/

in fact you can see it best in the logs when looking for 'Extracting templates from packages'. in the slow build this step took about 45 minutes, in the fast build it took like 2...

from what I can tell it slows down installation all around in all parts of apt, it just happens to be that batch template extraction is pretty much only opening and closing files thus being affected the most by the too high ulimit.

aleixpol commented 9 years ago

Are you sure this is necessary?

plasmamobile@b4b4912fca3c:~$ ulimit
unlimited
hsitter commented 9 years ago

root@1b450fec01a5:/# ulimit -n 1048576

aleixpol commented 9 years ago

So the problem is that it's too big? return -ENOSENSE;

hsitter commented 9 years ago

yes, I known, right!

I am not sure why you are surprised though, you have seen apt source code have you not? :P