Python parallel build fail

TinoDidriksen commented 5 years ago

@Vaydheesh (maybe @unhammer can help?), commit https://github.com/apertium/lttoolbox/commit/0fd248f8fe00c77357cab6a1c59f8368ca0fc30f fails when building in parallel (make -j4) with error:

Making all in python
make[1]: Entering directory '/misc/lttoolbox/python'
/usr/bin/python3 setup.py build
make[1]: *** No rule to make target 'lttoolbox.py', needed by 'all'.  Stop.

Works fine when re-run or in serial (make -j1), but need it to work in parallel from a pristine clone.

singh-lokendra commented 5 years ago

The issue is caused due to unnecessary BUILT_SOURCES lttoolbox.py. When make is run in series or re-run, the lttoolbox.py is already created by setup.py. Removing lttoolbox.py from BUILT_SOURCES seems to fix this issue.

unhammer commented 5 years ago

libdivvun solves this with pattern rules, which are pretty much the only way to have multiple targets in a parallel-safe way in gnu make. In the below rule, % and $* get replaced with libdivvun, the trick is to have two %'s on the left of the ::

%_wrap.cpp %.py: %.i setup.py std_unique_ptr.i ../src/libdivvun.la
    $(SWIG) -c++ -python -I$(top_srcdir)/src/lib -o $*_wrap.cpp -outdir . $(srcdir)/$*.i
    CPPFLAGS="$(CPPFLAGS)" CXXFLAGS="$(CXXFLAGS)" LDFLAGS="-Wl,-rpath,${prefix}/lib $(LDFLAGS)" $(PYTHON) setup.py build
# rpath in LDFLAGS so users don't have to set LD_LIBRARY_PATH

https://github.com/divvun/libdivvun/blob/b0c48d65bbee82341791329e82ec5bb5e55fdd87/python/Makefile.am#L11..L13

apertium / lttoolbox

Python parallel build fail #63