Error when building with make -j2

[firefly2442]

Testing master branch: 839e2ca1b4b36ac861efd125b8f9205270ba03ef

Default CMake options, LINK_AGAINST_INTERNAL_FFMPEG = TRUE

Creating config.mak and config.h...
make[3]: warning: jobserver unavailable: using -j1.  Add `+' to parent make rule.
make[3]: warning: jobserver unavailable: using -j1.  Add `+' to parent make rule.
make[3]: warning: -jN forced in submake: disabling jobserver mode.
CC  libavdevice/alldevices.o
CC  libavdevice/avdevice.o
CC  libavdevice/dv1394.o
make[3]: *** read jobs pipe: No such file or directory.  Stop.
make[3]: *** Waiting for unfinished jobs....
CC  libavdevice/fbdev.o
make[3]: *** write jobserver: Bad file descriptor.  Stop.
*** an error occured, aborting.
make[2]: *** [FFmpeg-binaries/lib/libavformat.so] Error 1
make[1]: *** [CMakeFiles/FFmpeg.dir/all] Error 2
make: *** [all] Error 2

[firefly2442]

https://www.gnu.org/software/make/manual/html_node/Error-Messages.html

I was running with make -j 2. When I ran just make, it compiled. Seems to be some threading issue?

[Ceylo]

Hmmm... what's currently done is that FFmpeg is always built with -j4. What happens if you edit jobsCount=4 in build_ffmpeg.sh with jobsCount=1 ?

[firefly2442]

That seems to fix it. Is that a good solution though? It will be forcing single-threading which may or may not be desirable... thoughts?

[Ceylo]

Hmmm.. the point is it'll be single threaded only for the sfeMovie core. For FFmpeg it's already using 4 processes at a time.

However what annoys me the most with the current solution is that the user is not able to easily choose how many processes should be used at the same time. So 4 may be too much or not enough.

What could be possible would be adding a CMake configuration entry for that. It would still NOT be used for the few sfeMovie files, but I could use that value when building FFmpeg. This doesn't fix your issue though. Anyone should be able to build with make -jx without issues…

[firefly2442]

https://github.com/Kitware/CMake/blob/1af0c41278567fa67e19d120b46058118f7a0d44/Modules/ProcessorCount.cmake

There is an official CMake module for finding the number of processors which could be used for your suggestion.

[Ceylo]

Thanks for the link!

Actually there are other issues with parallel builds so I'm wondering whether I should simply disable this feature: when building on Windows from Visual Studio, the build will simply stall with make.exe running at 100% CPU of one core and never stopping.

[Ceylo]

Parallel build of FFmpeg on Windows has been disabled (some time ago actually), so there should be no more build issues on Windows as long as the user doesn't use make -j*.

As for the make -j* issue. I don't really know what to do for now. I guess I'll have to ask Make developers :) Edit: oh no, a simple solution is to launch make instead of make -j4 for FFmpeg when the user has selected Makefiles build generator in CMake!

[Ceylo]

This should be fixed now:

• I let make handle parallelism for FFmpeg submake so you should no more have any build issue
• Parallel build is always disabled on Windows
• In other cases as many builds as the count of processors are done at the same time

Can you tell me if everything goes fine on your side?

[Ceylo]

Hmm doesn't seem to work well actually. Tested with make now. Regex for Makefile generator was wrong. But then make doesn't look to transmit the -j* option to submake…

Edit: at least it always work, but you won't benefit from parallel builds with Make… Edit 2: the only thing that looks to prevent the submake from correctly doing parallel builds is that there is a shell script between both: make executes build_ffmpeg.sh which then calls make.

[Ceylo]

I don't think I can change the workflow so that there is no more script between the main make and submake. So as I've no other solution, until there is some more information about this topic, I'll close this as "won't fix".