Brewing Lilypond --with-docs fails under 10.7.5

[yrammos]

Raring to brew, as always.

Here's the install.log and the config.log.

I get a SIG11 (segmentation fault) while compiling the docs, consistently at the same point in the process after many attempts. My hardware is well-behaving and I never get kernel panics or anything similar (i.e. not a hardware-induced fault).

I did not have the issue a few months ago under 10.7.4 and the same Lilypond version.

It seems to me possible that the lilypond binaries produced in this brew are not of high quality. But why?

[adamv]

Good question.

[2bits]

EDIT: To save everyone having to read 40 comments... lilypond is missing a dependency on fondu and env :std for 10.7. That fixes things pretty much.

---

Building docs is a latex thing, iirc, that is sensitive to 3rd party python modules you might have since installed. A sig 11 is a software error telling you that there was some unexpected code clash, not a kernel panic as you mentioned. So don't so much think about the exact error as decide whether you are willing to minimize the variables and move /usr/local to /usr/local.bak and reinstall. I used MacTex 2012 when I was testing, and I'm fairly confident you can succeed here, eventually.

[yrammos]

I just meant that SIG11 sometimes signifies a hardware fault, usually memory-related, but I can rule this possibility out in this case as I have been facing no other hardware-related issues such as kernel panics.

Building the Lilypond docs actually invokes Lilypond itself, typesetting tons of music examples.

As you may see in the install.log (now fixed—previously it was truncated, sorry) the failing command is actually an invocation of lilypond not of latex.

I have a very minimal installation of Python, basically only fontforge and dblatex (both required for Lilypond).

(530) (0) (lilydblatex) ▼  /usr/local/lib/python2.7/site-packages>  ls
total 96
drwxr-xr-x   5 rammos  admin   170 Nov  3 20:29 dblatex-0.3.4-py2.7.egg
drwxr-xr-x  10 rammos  admin   340 Nov  4 22:08 distribute-0.6.28-py2.7.egg
-rw-r--r--   1 rammos  admin   263 Nov  3 20:29 easy-install.pth
-rw-r--r--   1 rammos  admin   223 Nov  3 20:17 fontforge-1.0-py2.7.egg-info
-rwxr-xr-x   1 rammos  admin  8916 Nov  3 20:17 fontforge.so
drwxr-xr-x   4 rammos  admin   136 Nov  4 22:08 pip-1.2.1-py2.7.egg
-rwxr-xr-x   1 rammos  admin  8900 Nov  3 20:17 psMat.so
-rw-r--r--   1 rammos  admin   144 Nov  2 22:37 setuptools-0.6c11-py2.7.egg-info
-rw-r--r--   1 rammos  admin    30 Nov  4 22:08 setuptools.pth
-rw-r--r--   1 rammos  admin  1224 Nov  4 22:07 sitecustomize.py
-rw-r--r--   1 rammos  admin   600 Nov  4 22:08 sitecustomize.pyc

[yrammos]

The offending command (triggering the SIG11) is:

command failed: /private/tmp/lilypond-Mdbj/lilypond-2.16.0/out/bin/lilypond -I ./ 
-I ./out-www -I ../../input -I /private/tmp/lilypond-Mdbj/lilypond-2.16.0/Documentation 
-I /private/tmp/lilypond-Mdbj/lilypond-2.16.0/Documentation/snippets/out -I ../../input/regression/ 
-I /private/tmp/lilypond-Mdbj/lilypond-2.16.0/Documentation/included/ 
-I /private/tmp/lilypond-Mdbj/lilypond-2.16.0/mf/out/ -I /private/tmp/lilypond-Mdbj/lilypond-2.16.0/mf/out/ 
-I /private/tmp/lilypond-Mdbj/lilypond-2.16.0/Documentation/pictures 
-I /private/tmp/lilypond-Mdbj/lilypond-2.16.0/Documentation/pictures/./out-www -dbackend=eps --formats=ps,png,pdf  
-dinclude-eps-fonts -dgs-load-fonts --header=doctitle --header=doctitlecs --header=doctitlede 
--header=doctitlees --header=doctitlefr --header=doctitlehu --header=doctitleit --header=doctitleja --header=doctitlenl 
--header=doctitlezh --header=texidoc --header=texidoccs --header=texidocde --header=texidoces --header=texidocfr 
--header=texidochu --header=texidocit --header=texidocja --header=texidocnl --header=texidoczh -dcheck-internal-types 
-ddump-signatures -danti-alias-factor=2 -I  "/private/tmp/lilypond-Mdbj/lilypond-2.16.0/out/lybook-db"  -I  "."  -I  ".."  
-I  "/private/tmp/lilypond-Mdbj/lilypond-2.16.0/input/regression"  -I  "./"  -I  "./out-www"  -I  "../../input"  
-I  "/private/tmp/lilypond-Mdbj/lilypond-2.16.0/Documentation"  -I  "/private/tmp/lilypond-Mdbj/lilypond-2.16.0/Documentation/snippets/out"  
-I  "../../input/regression/"  -I  "/private/tmp/lilypond-Mdbj/lilypond-2.16.0/Documentation/included/" 
 -I  "/private/tmp/lilypond-Mdbj/lilypond-2.16.0/mf/out/"  -I  "/private/tmp/lilypond-Mdbj/lilypond-2.16.0/mf/out/" 
 -I  "/private/tmp/lilypond-Mdbj/lilypond-2.16.0/Documentation/pictures"  
-I  "/private/tmp/lilypond-Mdbj/lilypond-2.16.0/Documentation/pictures/./out-www" --formats=eps  -deps-box-padding=3.000000  
-dread-file-list -dno-strip-output-dir  "/private/tmp/lilypond-Mdbj/lilypond-2.16.0/out/lybook-db/snippet-names-9041008349041401427.ly"
Child was terminated by signal 11
Error ignored by lilylib
Error trapped by lilypond-book

Notably, just before this there is a very similar invocation, which completes without problems.

/private/tmp/lilypond-Mdbj/lilypond-2.16.0/out/bin/lilypond -dbackend=eps --formats=ps,png,pdf  -dinclude-eps-fonts 
-dgs-load-fonts --header=doctitle --header=doctitlecs --header=doctitlede --header=doctitlees --header=doctitlefr 
--header=doctitlehu --header=doctitleit --header=doctitleja --header=doctitlenl --header=doctitlezh --header=texidoc 
--header=texidoccs --header=texidocde --header=texidoces --header=texidocfr --header=texidochu --header=texidocit 
-header=texidocja --header=texidocnl --header=texidoczh -dcheck-internal-types -ddump-signatures -danti-alias-factor=2' 
--output=./out-www --format=texi-html --loglevel=WARN  --lily-output-dir /private/tmp/lilypond-Mdbj/lilypond-2.16.0/out/lybook-db 
--redirect-lilypond-output AAA-intro-regression.tely

Basically, the two invocations differ only in the source file they typeset and the search dirs (-I arguments).

The error is trapped by lilypond-book because lilypond is executed within the lilypond-book Python script.

And this is as far as I can get at the moment...

[2bits]

Do you have a non USA locale or anything? Try running the command manually. Any more info then? Maybe add a -v to the command if it accepts it. Perhaps you will get more info if you run that command through gdb.

dblatex cause ghc to sig11 during the building of it's docs, and I only had dblatex installed because I used it to build lilypond. So perhaps you can update your dblatex or reinstall it to be sure it's not the problem.

[2bits]

btw if you surround your terminal output with three backticks, it will all go into one easy to read code block.

For example:

Here is some text. And another line.

would create this:

Here is some text.
And another line.

[yrammos]

And the backticks need to be on their own line (which is what I forgot to do).

I can't run the offending command manually because the entire folder structure is temporary and deleted before the installer exits. I could check the tarball and see if that would work but I doubt it. Any way I can keep that temp folder and clean it up manually?

dblatex is in several ways quirky. For example it can only be installed with easy_install, not pip.

I do have a non-English locale. But I had it also last time I brewed the same ver of Lilypond+docs on 10.7.4 without problems. I know, this doesn't prove much but…

[2bits]

I prefer the long single line, but that's just me. Please gist your brew --config and brew doctor and brew list outputs.

You can shell out into the temporary directory when the install errors with

brew install -v --with-doc lilypond

It should bring up a menu of 5 choices. The last one is "Shell." When you shell out after the error, please gist the output of env.

You leave the temporary bash shell with exit 0 to finish a successful manual install, or exit 1 if you want to bail out gracefully.

If you could build Lilypond with docs previously, that's important. It's likely if you just remove and reinstall everything that this will work, but we should look at your gists of course.

sudo mv /usr/local /usr/local.bak
sudo mkdir /usr/local
sudo chown $USER:admin /usr/local
chmod 775 /usr/local
cd /usr/local
then run the Homebrew install

I'll try to build this on my Lion setup in the meantime.

[yrammos]

Will do all that. Meanwhile, please note that you need --env=std (the latter on my setup at least) to build the lilypond binaries.

[2bits]

It looks like the -d option is broken by an internal Homebrew Readline related bug I reported last week. We won't be able to use that. I've been getting different build errors before I get to the docs part that I'm trying to sort out with --env=std and updating my OS.

[2bits]

You should read this: https://github.com/mxcl/homebrew/issues/14559

[yrammos]

I am actually the author of this :-)

[2bits]

Oh so you patched /usr/include/FlexLexer.h and get past make. I'll try to get to that point tomorrow. I was thinking all we needed was to #include <stddef.h> to get that size_t defined.

[yrammos]

Perhaps I used a sledgehammer for a tiny nut but I thought to patch it as the FlexLexer.h file shipped with 10.8 has int replaced with size_t. So does the latest version upstream. Though it does make me a bit anxious to touch system libraries.

On 6 Nov 2012, at 11:27, 2bits notifications@github.com wrote:

Oh so you patched /usr/include/FlexLexer.h and get past make. I'll try to get to that point tomorrow. I was thinking all we needed was to #include to get that size_t defined.

— Reply to this email directly or view it on GitHub.

[2bits]

I forget, did this issue make it upstream? Do you have link to the bug report?

[yrammos]

I did file it but it was more or less dismissed as uninteresting (my words). Upstream seems content with distributing binaries, I think, and is not very concerned with debugging individual build systems.

On 6 Nov 2012, at 11:46, 2bits notifications@github.com wrote:

I forget, did this issue make it upstream? Do you have link to the bug report?

— Reply to this email directly or view it on GitHub.

[2bits]

Ok I'll try again. We might need to brew flex from source and add it as a dependency. Working on this a bit as time permits, as there is no flex formula yet because it would be a dupe.

[yrammos]

Yes, it would be ideal to include the latest version available at http://flex.sourceforge.net, overriding the version shipped with the OS.

[yrammos]

Inspired by some similar bug reports (from older versions of Lilypond), and thinking that this is a somewhat elusive fluke, I modified the formula to build the docs one process (and on a single CPU) at a time:

make -j1 CPU_COUNT=1 doc

instead of the plain

make doc

No success, unfortunately, so we can rule out the possibility that this was due to CPU congestion.

[2bits]

I'll try to boot into 10.7 today and work on this.

[yrammos]

Here is my brew --config:

HOMEBREW_VERSION: 0.9.3
HEAD: b5942ecb523da69ae16ef71b57653fccb137be35
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CELLAR: /usr/local/Cellar
CPU: dual-core 64-bit penryn
OS X: 10.7.5-i386
Xcode: 4.5.2
CLT: 4.5.0.0.1.1249367152
GCC-4.2: build 5666
LLVM-GCC: build 2336
Clang: 4.1 build 421
X11: 2.6.4 => /usr/X11
System Ruby: 1.8.7-358
Perl: /usr/bin/perl
Python: /usr/local/bin/python => /usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/2.7/bin/python2.7
Ruby: /Users/rammos/.rvm/rubies/ruby-1.9.3-p286/bin/ruby

[2bits]

Yea I figured it out by the fact that dblatex won't install unless you have HB python and by reading your install.log. It's early here :-)

[yrammos]

Just to confirm that the SIG11 issue persists on my system after updating the formula to the newly released 2.16.1 version.

(Pull request: https://github.com/mxcl/homebrew/issues/15959)

[2bits]

The fontforge formula is broken. I had to fix it to depend on libpng so that it can convert woff fonts. make install was not working for me because of that.

We might need to throw env=std for everyone for lilypond. Don't know yet.

I'm curious if you set that docbook env var XML_CATALOG_FILES like the docbook caveats say to do before you tried to brew lilypond?

[2bits]

I just reproduced the Sig 11 error you are getting.

[yrammos]

Yeah it is still set in my .bashrc from a previous install.

Strange that you couldn't make install without tweaking fontforge. My system didn't stumble on that...

[yrammos]

Are you on 10.7.4 or 10.7.5?

And may I ask what your CPU is?

[2bits]

You probably had libpng and jpeg and libtiff installed already when you built fontforge. I purposely wiped all my brews first to find these issues. I'm duplicating your setup on 10.7.4. I have no updates available from the App Store.

[yrammos]

Actually I'm on 10.7.5 but good to know you're on 10.7.4 as this rules out the OS update as a possible culprit.

[2bits]

Did you get 10.7.5 from the App Store as a regular user?

[yrammos]

No, I got it via Software Update in the OS. Neither an App Store update nor a Combo download from the Apple website.

[2bits]

I've been digging a bit into the error this issue is about,

command failed: /private/tmp/lilypond-J0Ua/lilypond-2.16.1/out/bin/lilypond -I ./ -I ./out-www -I ../../input -I /private/tmp/lilypond-J0Ua/lilypond-2.16.1/Documentation -I /private/tmp/lilypond-J0Ua/lilypond-2.16.1/Documentation/snippets/out -I ../../input/regression/ -I /private/tmp/lilypond-J0Ua/lilypond-2.16.1/Documentation/included/ -I /private/tmp/lilypond-J0Ua/lilypond-2.16.1/mf/out/ -I /private/tmp/lilypond-J0Ua/lilypond-2.16.1/mf/out/ -I /private/tmp/lilypond-J0Ua/lilypond-2.16.1/Documentation/pictures -I /private/tmp/lilypond-J0Ua/lilypond-2.16.1/Documentation/pictures/./out-www -dbackend=eps --formats=ps,png,pdf  -dinclude-eps-fonts -dgs-load-fonts --header=doctitle --header=doctitlecs --header=doctitlede --header=doctitlees --header=doctitlefr --header=doctitlehu --header=doctitleit --header=doctitleja --header=doctitlenl --header=doctitlezh --header=texidoc --header=texidoccs --header=texidocde --header=texidoces --header=texidocfr --header=texidochu --header=texidocit --header=texidocja --header=texidocnl --header=texidoczh -dcheck-internal-types -ddump-signatures -danti-alias-factor=2 -I  "/private/tmp/lilypond-J0Ua/lilypond-2.16.1/out/lybook-db"  -I  "."  -I  ".."  -I  "/private/tmp/lilypond-J0Ua/lilypond-2.16.1/input/regression"  -I  "./"  -I  "./out-www"  -I  "../../input"  -I  "/private/tmp/lilypond-J0Ua/lilypond-2.16.1/Documentation"  -I  "/private/tmp/lilypond-J0Ua/lilypond-2.16.1/Documentation/snippets/out"  -I  "../../input/regression/"  -I  "/private/tmp/lilypond-J0Ua/lilypond-2.16.1/Documentation/included/"  -I  "/private/tmp/lilypond-J0Ua/lilypond-2.16.1/mf/out/"  -I  "/private/tmp/lilypond-J0Ua/lilypond-2.16.1/mf/out/"  -I  "/private/tmp/lilypond-J0Ua/lilypond-2.16.1/Documentation/pictures"  -I  "/private/tmp/lilypond-J0Ua/lilypond-2.16.1/Documentation/pictures/./out-www" --formats=eps  -deps-box-padding=3.000000  -dread-file-list -dno-strip-output-dir  "/private/tmp/lilypond-J0Ua/lilypond-2.16.1/out/lybook-db/snippet-names--1159634814230049487.ly"
Child was terminated by signal 11
Error ignored by lilylib
Error trapped by lilypond-book

Please see /private/tmp/lilypond-J0Ua/lilypond-2.16.1/out/lybook-db/snippet-names--1159634814230049487.log

make[3]: *** [out-www/collated-files.texi] Error 1

and I've been investigating collated-files.texi, which is part of a regression test, I think: http://lilypond.org/doc/v2.17/Documentation/contributor-big-page#regression-tests There is no configure option to turn off building of tests.

Still investigating, but it's hard because I have no -d to shell out into the source after error.

[2bits]

Using a

begin
  system 'make docs'
rescue
  system '/usr/local/bin/smultron', 'out/lybook-db/snippet-names--1159634814230049487.log'
end

I believe I found the problem was simply a missing dependency on cheese:

Renaming input to: `markup-special-characters.ly'
Interpreting music...[8]
Preprocessing graphical objects...
Calculating line breaks... 
Drawing systems... 
Writing header field `texidoc' to `./19/lily-294a1fa9.texidoc'...
Writing ./19/lily-294a1fa9-1.signature
Writing ./19/lily-294a1fa9-2.signature
Layout output to `./19/lily-294a1fa9.eps'...failed (0): fondu: Failed to execute child process "fondu" (No such file or directory)

Homebrew has a formula for fondu. I'm going to add that and a couple of other tweaks if you don't mind closing your pull request for 2.16.1. I don't want to have to start over or get a clash merging.

[yrammos]

That's great. So did the -d option work after all? How did you shell out?

The 2.16.1 pull request has been already merged.

[2bits]

I didn't use -d or try to shell out. I used a ruby rescue clause to catch the error and launch my text editor so I could examine the log file.

[2bits]

You might as well add a depends_on 'fondu' to your local copy and start testing.

[yrammos]

I am adding this as a global dependency, not specific to --with-doc.

[2bits]

Me too.

[yrammos]

First signs very encouraging! It went past the SIG11 trigger. Still compiling.

[2bits]

Mine finished just now. Total compile time 79.3 min

[yrammos]

104.6 minutes here. Though to my ageing Macbook's defense, I was LaTeX-ing intensively in the foreground...

Many thanks for this :+1:

I'm filing a pull request for the fondu dependency.

[yrammos]

Pull request https://github.com/mxcl/homebrew/issues/15964 filed. Congratulations and thanks!