Build on Mac OS X problems

[esd100]

So, I just had a post about building Julia with MKL and Intel Compilers, which never really got resolved (see #6239 ).

So, I went to the extreme and decided to wipe my system clean and start from a fresh. Nothing has been installed except gfortran 4.9 and XCode 5.1 (which includes command line tools). So, this is an absolutely bare-bones, super clean system, with what I believe to be is just the required software for building.

I decided against using MKL and intel compilers for this experiment, just to see if I could build Julia properly using the basics. Unfortunately, building failed.

Here are the specs for my system: Processor: 2.6 GHz Intel Core i7 Haswell Memory: 16 GB 1600 MHz DDR3 Graphics: Intel Iris Pro 1024 MB Software: OS X 10.9.2 (13C64) Compiler: Apple LLVM version 5.1 (clang-503.0.38) (based on LLVM 3.4svn) Target: x86_64-apple-darwin13.1.0 Thread model: posix Versions.make file: OPENBLAS_VER = develop Make.user file: OPENBLAS_TARGET_ARCH=HASWELL USE_SYSTEM_LLVM=1 CXXFLAGS=-O3 FFLAGS=-O3

As I said, the build failed with this error. checking whether the Fortran 77 compiler works... no configure: error: in /Users/esd100/julia/deps/arpack-ng-3.1.3' configure: error: Fortran 77 compiler cannot create executables See config.log for more details make[2]: * [arpack-ng-3.1.3/config.status] Error 77 make[1]: * [julia-release] Error 2 make: *\ [release] Error 2

Building took a rather long time and the terminal output is more than 100 pages long, so I just decided to print the terminal output to a pdf, upload it to my google drive and share it for further evaluation by the Julia developers. https://drive.google.com/file/d/0By8PHgo9jn1PaVd3dFRLNkQ0MlE/edit?usp=sharing

[jiahao]

You'll have to change the permissions so that we can actually see the output.

[StefanKarpinski]

How about posting that output as a gist instead of a PDF?

[esd100]

Does this work? https://gist.github.com/esd100/9725270

[jiahao]

Yes. The error log refers to the file deps/arpack-ng-3.1.3/config.log, which would be useful to post also.

[esd100]

This is the config.log for arpack. https://gist.github.com/esd100/9725480

[staticfloat]

How did you install gfortran? It looks like your gfortran installation is broken somehow.

[jiahao]

@staticfloat beat me to it. This line in the gist says that gfortran could not find the C runtime library, which shouldn't happen to a correctly installed compiler. Are you using gfortran from hpc.sourceforge.net by any chance? Builds from that website have a long history of causing problems like this.

[esd100]

Yes, actually. I was using the gfortran from hpc. Should I try to point it to the correct C runtime library or clean and rebuild with source from gnu gcc? And, depending on what the answer to previous question, how would I go about doing that?

[jiahao]

I personally use gfortran installed through Homebrew and so do many other Julia developers on their Macs, so I would recommend you try it. It's a one-line install for homebrew itself, and then brew install gfortran for gfortran. Homebrew tries to match the version of gfortran with the version of gcc installed by your XCode, which in their experience eliminates most of these kinds of incompatibility issues.

[staticfloat]

Additionally, the homebrew install is really quick; it downloads a precompiled binary in this case. You will probably have to uninstall your hpc gfortran though. On Mar 23, 2014 10:06 AM, "Jiahao Chen" notifications@github.com wrote:

I personally use gfortran installed through Homebrew http://brew.sh/and so do most of the other Julia developers, so I would recommend you try it. It's a one-line install for homebrew itself, and then brew install gfortran for gfortran. Homebrew tries to match the version of gfortran with the version of gcc installed by your XCode, which in their experience eliminates most of these kinds of incompatibility issues.

Reply to this email directly or view it on GitHubhttps://github.com/JuliaLang/julia/issues/6244#issuecomment-38388515 .

[esd100]

I don't know if this will effect your approach or not, but Apple has eliminated GCC from their XCode toolchain (https://en.wikipedia.org/wiki/Xcode).

[jiahao]

What I said about version matching is probably only of historical interest, but it's still true that homebrew's compilers work much more nicely than hpc's. And yes, you will want to scrub your system of the hpc compiler.

[esd100]

This post from enthought developer mentions that homebrew installations should not be done in mavericks (https://support.enthought.com/entries/26184115-GCC-Clang-and-Cython-in-OS-X-10-9-Mavericks).

Here is a post on stack-exchange discussing the topic of gcc pointing to clang in OS X 10.9 and XCode 5.1 (https://stackoverflow.com/questions/19535422/os-x-10-9-gcc-links-to-clang)

I'm wondering if I should just try to point the current installation to the c runtime library.

[jiahao]

If you ask for help, the least you could do is have the good graces to try what is suggested. Otherwise that just wastes everyone's time.

[esd100]

Just to review, as I haven't done a cleaning of an installation in awhile. The gFortran from HPC was just to unzip/extract to the package to usr/local/. There was nothing else done, per se, in terms of 'make' or 'install'. So, how would I clean the package?

[esd100]

I appreciate your help but I don't have good graces.

[esd100]

I have seen many posts on the net about not using homebrew with mavericks. So, in my mind, it is foolish to not proceed with caution.

[esd100]

The least of all issues is that Julia is not 'future' compatible with a pristine, late 2013, popular and extremely powerful machine, without an inane amount of debugging. It's unfortunate. I'm not doing this because I need to, nor do I need your help. I'm doing it because I am interested in doing it. I don't have any time constraints or other pressures that require me to install 'macports' or 'homebrew', which are notorious for breaking things.

[esd100]

And, if you are a developer of Julia, you are certainly rude. To think that you would reject the opportunity to use a public 'beta-tester' on a pristine, new machine, when your source code does not compile correctly, makes me think you are more controlling than helpful.

[Keno]

You are probably missing the command line tools. Try xcode-select --install.

[johnmyleswhite]

FWIW, I was able to install Julia on a pristine 10.9 machine without using Homebrew about a month ago.

[esd100]

@loladiro, nope. Those are installed by default with XCode 5.1

[Keno]

Last I checked, they were not. Regardless, could you post the output of that command?

[esd100]

Lol...I had to look up what FWIW meant. Well, that is certainly interesting. @johnmyleswhite . I'm wondering what the differences are. I like the idea of focusing on root cause analysis and resolution. Maybe we could figure out why the linking is failing and help prevent it from happening again.

[esd100]

@loladiro, yes, this is new packaging of XCode as of 5.1, which was released just a couple weeks ago. So, you may not have seen the newest packaging. One benefit of the new packaging environment, is that you can install the command line tools without installing XCode, if you do not plan on doing any development but just want the tools.

[Keno]

Please, just post the output of that command. If I read the release notes correctly that command is still needed to adjust liker search paths.

[esd100]

@loladiro, if the command is put into the terminal, a pop up window does ask you to install or install XCode with tools. While this may lead you to believe the CLT are not there, they are in fact there. I went through this process earlier, which is why I know this. That is why if you type in certain commands like clang -version, you get display output. If the CLT weren't installed, there would be no way for the terminal to know that command and it would give an error. xcode-select -v --> xcode-select version 2333.

For example, 'clang - -march=native -E -v - </dev/null 2>&1 | grep cc1 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.9.0 -E -disable-free -disable-llvm-verifier -main-file-name - -mrelocation-model pic -pic-level 2 -mdisable-fp-elim -masm-verbose -munwind-tables -target-cpu x86-64 -target-linker-version 236.3 -v -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk -fdebug-compilation-dir /Users/esd100 -ferror-limit 19 -fmessage-length 0 -stack-protector 1 -mstackrealign -fblocks -fobjc-runtime=macosx-10.9.0 -fencode-extended-block-signature -fdiagnostics-show-option -vectorize-slp -o - -x c - clang -cc1 version 5.1 based upon LLVM 3.4svn default target x86_64-apple-darwin13.1.0 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.9.0 -E -disable-free -disable-llvm-verifier -main-file-name - -mrelocation-model pic -pic-level 2 -mdisable-fp-elim -masm-verbose -munwind-tables -target-cpu x86-64 -target-linker-version 236.3 -v -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/5.1 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk -fdebug-compilation-dir /Users/esd100 -ferror-limit 19 -fmessage-length 0 -stack-protector 1 -mstackrealign -fblocks -fobjc-runtime=macosx-10.9.0 -fencode-extended-block-signature -fdiagnostics-show-option -vectorize-slp -o - -x c - clang -cc1 version 5.1 based upon LLVM 3.4svn default target x86_64-apple-darwin13.1.0 '

[esd100]

[Keno]

Ok, just making sure. In that case, the only thing I can think of is that the gfortran on hpc.sourceforge has not been updated to build against XCode 5.1, so I join everybody else in recommending homebrew (which works just fine for me on 10.9). You can of course also build gfortran from source, but that's a lot of pain and probably wouldn't work since you have to be quite careful to get the sysroot right (I know this from building clang from source all the time).

[esd100]

This is the window that pops up with the xcode-select --install command.

[esd100]

But you can see that there is an inconsistency. It says in the dialog box, "xcode-select" command requires CLD tools. So, if you click "Not Now", then type "xcode-select" into terminal you should get an error. But you don't, as you can see in the following image.

[esd100]

ok, well, I will continue to search for an answer to the root cause of these problems. Maybe it is that hpc has not built against 5.1, maybe not. I will consider using homebrew as well. I think I will manually have to clean my usr/bin/ folder from the provided hpc package.

[esd100]

I found a similar problem, although quite dated, at the following link. I'm not sure if this is related in causality. Either way, I like the way they summarized the problem: Problem / Environment / Root Cause / Resolution. http://software.intel.com/en-us/articles/library-not-found-for-lcrt1105o

[tkelman]

@esd100 quite likely. Can the version of gfortran that you downloaded compile a simple "Hello world" Fortran program? Say

       program hello
          print *, "Hello World!"
       end program hello

? If not, I will join the crowd in recommending Homebrew's version of gfortran. Most troubles with Mavericks are due to Apple's decision to change the default C++ standard library, so you do have to be careful with that (and the Homebrew developers have worked tirelessly for the past 6 months to make sure they are doing the right thing in as many cases as possible).

Where you get your Fortran compiler from is mostly orthogonal to that concern, and since Apple does not ship a Fortran compiler with XCode, job 1 is finding a known working version of gfortran. If you insist on avoiding Homebrew, then you're on your own for finding an alternate version. Let us know once you are sure that your gfortran version of choice is at least functioning properly.

Regarding why the HPC version of gfortran doesn't work, it was probably built on an older machine that was upgraded to 10.9 rather than a clean installation, so it still has some previous SDK versions lying around in /Developer/SDKs/ and was hard-coding library search paths to look in locations that don't exist on other machines. This is a serious bug that should be reported to whoever is responsible for building the HPC gfortran, if you want that version to be usable in the future.