Building and pushing uncompressed images

berkeley-dsep-infra / hubploy

Toolkit to deploy many z2jh based JupyterHubs

BSD 3-Clause "New" or "Revised" License

17 stars 15 forks source link

Building and pushing uncompressed images #38

Open tjcrone opened 4 years ago

tjcrone commented 4 years ago

I have been trying to figure out ways to reduce the time it takes to spin up new nodes when they are required to accommodate new users. I have noticed that most of the time for pulling images (at least in our case) is taken up by layer decompression. Do we know if Docker has a way of building images with layers that are not compressed? Is this functionality exposed in r2d so we can expose it in hubploy? Or is this simply not possible? Thanks!

yuvipanda commented 4 years ago

Good question! I'm not sure. Decompression is the longest phase in our case too, but we nailed it down to disk IO being slow. Switching to SSD base disks made it so much faster. So maybe that would be useful?