search

containers / ai-lab-recipes

Examples for building and running LLM services and applications locally with Podman
Apache License 2.0
103 stars 106 forks source link

Fix long container startup times #763

Closed n1hility closed 1 month ago

n1hility commented 1 month ago

The use of a uid map leads to a new layer with all files chowned. This takes several seconds due to the size of the instructlab container (26GB). Normally this would be a one time cost where the idmap layer is cached and reusued accross container creations; however, since the container is stored on a read-only additional image store, no caching is performed.

Address the problem by creating a derived empty contianer in mutable container storage. This allows the 1k idmap layer to be created in the smae area, yet reuses the layers in additional image store.

n1hility commented 1 month ago

PTAL @Gregory-Pereira @lmilbaum , another high priority fix

// cc @rhatdan

rhatdan commented 1 month ago

LGTM @giuseppe @nalind PTAL

cgwalters commented 1 month ago

Backing up a second, we should support kernel native idmapping and shouldn't need to chown I think...

rhatdan commented 1 month ago

I don't think idmapping was supported in RHEL for 9.4.

n1hility commented 1 month ago

Right yeah we are on 5.14 and this was added to overlay in 5.19 right?

rhatdan commented 1 month ago

Not sure of the numbers but sounds right.

cgwalters commented 1 month ago

OK yes I guess overlayfs idmapping is only going to be for RHEL10 so far unless further backports happen.