clearlinux / distribution

Placeholder repository to allow filing of general bugs/issues/etc against the Clear Linux OS for Intel Architecture linux distribution
521 stars 29 forks source link

Mixer, build update fails with more workers. #2490

Open postmannen opened 2 years ago

postmannen commented 2 years ago

Hi.

My mix server virtual machine currently got 6 cores.

If I create and init a new mix, do a mixer build bundles, and then do a mixer build update it starts up 6 workers, and it fails with errors like: Error: Couldn't build update: couldn't make pack linux-firmware for version 10: write |1: broken pipe.

The same have a working mix at X version, and add a new package, it complains when updating with for example: Error: Couldn't build update: couldn't make pack lib-opengl for version 50: write |1: broken pipe

But, if I force the number of workers to 1 with: mixer build update --log-level=5 --bundle-workers=1 --delta-workers=1 --fullfile-workers=1 Then it works.

So it seems the problem is only when working with number of workers >1.

Can it be something like the mixer when using several workers concurrently they mess up for each other ? Like a synchronization issue between the workers working on for example with/on the same set of files ?

bryteise commented 2 years ago

What is the open file descriptor limit you have set? I haven't seen an issue creating mixes with up to 56 cores locally and I think we use more when building the releases.

postmannen commented 2 years ago

It is at it's default. I have not touched the limits.

Here are the values.


> sysctl -n fs.file-max
9223372036854775807

> ulimit -Hn
524288

> ulimit -Sn
1024
```