search

coreos / coreos-assembler

Tooling container to assemble CoreOS-like systems
https://coreos.github.io/coreos-assembler/
Apache License 2.0
347 stars 168 forks source link

bump cache qcow size; skip compression on manifest.json #3942

Closed dustymabe closed 2 weeks ago

dustymabe commented 2 weeks ago
dustymabe commented 2 weeks ago

cmdlib: bump cache size to 30G

Not sure if there will be fallout from this or not.

dustymabe commented 2 weeks ago

Did miss something in the commit message? It only has With

Fun. Yes - there is a thought that is missing there.

It should be:

cmdlib: bump cache size to 30G

With us now building more in OSBuild in a single run [1] we need
a larger cache2.qcow2 to hold more pipeline outputs concurrently.

[1] https://github.com/coreos/fedora-coreos-pipeline/pull/1055

If this looks good I'll push up that change and merge it.

jlebon commented 2 weeks ago

Ouch. This is getting kinda awkwardly large. Hmm, are we storing artifacts twice in the cache qcow2 due to the "musical chair" bit we have there? Feels like instead we should just change osbuild's cp -a when exporting things be more lax wrt preserving attributes?

Anyway, LGTM!

dustymabe commented 2 weeks ago

Ouch. This is getting kinda awkwardly large. Hmm, are we storing artifacts twice in the cache qcow2 due to the "musical chair" bit we have there?

Theoretically we aren't storing artifacts twice in the cache qcow2 because it reflinks things between the store and the outdir. We do just have a lot of pipelines/stages that copy a largeish image from one stage to another and then makes a small change to it.. when we make that small change I think we lose the reflink and it takes up more space now??

Feels like instead we should just change osbuild's cp -a when exporting things be more lax wrt preserving attributes?

Yeah. Maybe we should just patch it? Though I think soon with idmapped virtioFS in kernel 6.12 we can solve that problem for ourselves.