Building cmark: wrong build order when using multiple cores

[zmwangx]

Simplified issue description per @tdsmith's comment:

When building cmark, the order of targets could be wrong when multiple cores are used. Steps to reproduce:

mkdir build && cd build
cmake ..
make -j4

Error:

Traceback (most recent call last):
  File "/Users/zmwang/software/CommonMark-0.15/man/make_man_page.py", line 24, in <module>
    cmark = CDLL("../src/libcmark.dylib")
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(../src/libcmark.dylib, 6): image not found
Building C object src/CMakeFiles/libcmark.dir/node.c.o
make[2]: *** [man/cmark.3] Error 1
make[1]: *** [man/CMakeFiles/man.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

This error occurred because the man target was built before libcmark.

---

Old issue description:

When building cmark in certain environments, the order of targets could be wrong, e.g., man before libcmark. I noticed this issue when I tried to build cmark using Homebrew (see Homebrew/homebrew #35735), but otherwise there was no problem. A Homebrew maintainer believe that this is not a Homebrew issue.

Steps to reproduce the issue:

• Grab a copy of commonmark.rb from Homebrew/homebrew #35735 (yet to be merged);

• Emulate Homebrew's build environment by

  brew install --interactive --debug --verbose commonmark

• In the new interactive session,

  mkdir build && cd build
  cmake ..
  make

• Error:

  Traceback (most recent call last):
  File "/tmp/commonmark-i1ydBw/CommonMark-0.15/man/make_man_page.py", line 24, in <module>
    cmark = CDLL("../src/libcmark.dylib")
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
  OSError: dlopen(../src/libcmark.dylib, 6): image not found
  make[2]: *** [man/cmark.3] Error 1
  make[1]: *** [man/CMakeFiles/man.dir/all] Error 2
  make[1]: *** Waiting for unfinished jobs....

  I investigated and realized that the error occurred because libcmark.dylib (clearly a product of the libcmark target) was not yet built when man/make_man_page.py was called (clearly for the man target).

  The right build order (when built outside Homebrew) is cmark, libcmark, api_test, and finally man.

• By the way, the issue can be bypassed in Homebrew by calling ENV.deparallelize before system "make".

---

Since I have little experience with CMake, I don't know what further information I should provide. I'm happy to provide more details when asked.

[tdsmith]

ENV.deparallelize runs make with -j1 (otherwise Homebrew invokes make with -j set to the number of processing cores) so the upshot is that commonmark's makefile can behave inelegantly when make's parallelism is enabled.

[zmwangx]

@tdsmith Thanks for the explanation, I simplified the issue description accordingly. (Somehow I didn't realize from the name what ENV.deparallelize actually does...)

[jgm]

Going forward, I'm just going to include cmark.3 in the repo and the archive, so we avoid these complications. But the patch I pushed to close this issue should work for you now.

[zmwangx]

@jgm Yes it does, thanks.