"make -j native-deb-utils native-deb-kmod" is broken

[rincebrain]

System information

Type	Version/Name
Distribution Name	Debian
Distribution Version	11
Kernel Version	doesn't matter
Architecture	arm64
OpenZFS Version	678a3b8f9

Describe the problem you're observing

Trying to do make -j native-deb-utils native-deb-kmod can break horribly, apparently because it attempts to do native-deb-utils and native-deb-kmod in parallel, and since they both want to use the same working directory, you get wild explosions that are different every time as they step on each other.

Describe how to reproduce the problem

• Grab recent git
• make -j native-deb-utils native-deb-kmod
• Bang.

Include any warning/errors/backtraces from the system logs

I tried it 5 times and got 5 different errors, among them, claiming my kernel had CONFIG_MODULES=n, unable to md5sum a temporary file in build/, and other such fun.

The two probably just need a rule to prevent them parallelizing like this, or to just build in different working trees, but.

[mcmilk]

@nabijaczleweli - have you some idea on this problem? I tried some things ... but I never got some nice result :/

[nabijaczleweli]

make -j native-deb-utils; make -j native-deb-kmod

I didn't recognise these rules, and that's because they seem to originate in e6e31dd5406d59edaaf5a7eeebd2fb83fb86236f and 4d631a509d067c2695a396983cb12f58d92d4e04 which is post-me-replacing-the-build-system, so no; and I tend to build the actual Debian packages. My opinion from before stands: I couldn't care less about random unpatched and unmaintained debs, and the only thing contrib/debian, rpm, and config/{rpm,deb}.am are good for is getting rm -red.

[mcmilk]

Ui, I didn't know that you liked this debian thing so much. Sorry ;-)

[rincebrain]

Hey @usaleem-ix, it seems like you might have noticed this during testing, since e6e31dd5406d59edaaf5a7eeebd2fb83fb86236f deletes the -j from the CI build rules...

[usaleem-ix]

Hi @rincebrain, IIRC -j was removed since parallel build is configured already from contrib/debian/rules.in, using -j prior in the make command breaks the build. To stay inline with Debian packaging bits, this was not modified:

NUM_CPUS = $(shell nproc 2>/dev/null)
PARALLEL = $(subst parallel=,,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
NJOBS    = -j$(or $(PARALLEL),$(NUM_CPUS),1)

make native-deb / make native-deb-utils / make native-deb-kmod should be running with max number of CPUs already and should not be invoked with -j option.

[rincebrain]

I don't think saying "don't use make -j" is a reasonable answer to "make -j is broken for the build targets you added".

[usaleem-ix]

Sorry, what I meant was, since for Debian packaging, parallel build is configured from rules.in, via DEB_BUILD_OPTIONS variable, it did not seem correct to override with -j flag initially. I prefer not to break that interface. I'll try and see how can both interfaces can work together.

[behlendorf]

The recently tagged 2.3.0-rc1 release included a couple of fixed. It'd be very helpful if anyone who had issues could try again so we can work through any potential removing issues. We'll backport these for 2.2.7.

[snajpa]

@usaleem-ix @behlendorf commit cc9e36a42 causes all the userspace components to build sequentially even when I'm not building debs at all - here's my build oneliner

git clean -fdx; ./autogen.sh; ./configure --enable-debug --with-linux=/root/linux && make -j64 && make install

[usaleem-ix]

@usaleem-ix @behlendorf commit cc9e36a causes all the userspace components to build sequentially even when I'm not building debs at all - here's my build oneliner

git clean -fdx; ./autogen.sh; ./configure --enable-debug --with-linux=/root/linux && make -j64 && make install

I'll look into this.