Failed build on FreeBSD

[forkandwait]

Hi all,

I followed the directions for building on FreeBSD, including installing libunwind, but I get an error. Any pointers would be appreciated. See below for some info.

My machine (more specs avail if important):

FreeBSD mustafa 9.0-RELEASE FreeBSD 9.0-RELEASE #0: Tue Jan  3 07:15:25 UTC 2012     root@obrian.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386

The most recent git commit in the log is e4ca3a9d13b7a501214a3673757ac8e28b6c4667:

commit e4ca3a9d13b7a501214a3673757ac8e28b6c4667
Author: Tim Holy <tim.holy@gmail.com>
Date:   Fri May 11 14:49:46 2012 -0500

gmake output:

julia/ (j=0,r=0)$ gmake
Submodule 'deps/libuv' (https://github.com/JuliaLang/libuv.git) registered for path 'deps/libuv'
Cloning into 'deps/libuv'...
remote: Counting objects: 10053, done.
remote: Compressing objects: 100% (2610/2610), done.
remote: Total 10053 (delta 7830), reused 9534 (delta 7390)
Receiving objects: 100% (10053/10053), 2.97 MiB | 376 KiB/s, done.
Resolving deltas: 100% (7830/7830), done.
Submodule path 'deps/libuv': checked out 'd48c5bd0adcb08fc8793529aed8a19530efde9ee'
In file included from src/unix/eio/eio.c:51:
src/unix/eio/ecb.h: In function 'ecb_unreachable':
src/unix/eio/ecb.h:333: warning: 'noreturn' function does return
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  245k  100  245k    0     0   105k      0  0:00:02  0:00:02 --:--:--  126k
Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|--- dSFMT.h    2012-01-01 11:56:06.000000000 +0530
|+++ dSFMT-julia.h  2012-01-01 11:55:19.000000000 +0530
--------------------------
Patching file dSFMT.h using Plan A...
Hunk #1 succeeded at 1 with fuzz 2.
Hunk #2 failed at 186.
Hunk #3 failed at 233.
Hunk #4 failed at 254.
Hunk #5 failed at 273.
Hunk #6 failed at 284.
Hunk #7 failed at 296.
Hunk #8 failed at 307.
Hunk #9 failed at 319.
Hunk #10 failed at 330.
Hunk #11 failed at 342.
Hunk #12 failed at 365.
Hunk #13 failed at 380.
Hunk #14 failed at 396.
Hunk #15 failed at 412.
Hunk #16 failed at 428.
Hunk #17 failed at 438.
Hunk #18 failed at 448.
Hunk #19 failed at 459.
Hunk #20 failed at 472.
Hunk #21 failed at 500.
Hunk #22 failed at 509.
Hunk #23 failed at 518.
Hunk #24 failed at 528.
Hunk #25 failed at 538.
Hunk #26 failed at 548.
Hunk #27 failed at 558.
Hunk #28 failed at 568.
Hunk #29 failed at 581.
Hunk #30 failed at 594.
Hunk #31 failed at 607.
Hunk #32 failed at 619.
31 out of 32 hunks failed--saving rejects to dSFMT.h.rej
done
gmake[2]: *** [random/jl_random.c] Error 31
gmake[1]: *** [julia-release] Error 2
gmake: *** [release] Error 2

[pao: formatting]

[Keno]

Seems like our patch for dSFMT.h is not being applied properly. Is there a functional difference between the patch command on Linux and that on FreeBSD?

[forkandwait]

There are both patch and gpatch on FreeBSD, I just discovered prompted by your question. Analogous to make and gmake, I guess. So, probably, yes.

I am not sure the best way to set this up -- I don't want to link patch to gpatch in the file system as root (it will probably mess a pile of stuff up), but I do want to make it work for the entire build of Julia.

Ideas?

[Keno]

Can you replace all calls to patch by gpatch in deps/Makefile. If that solves your issue, I'll push a fix to master that automatically uses the right tool.

[forkandwait]

Changed as recommended, get the following errors:

julia/ (j=0,r=0)$ gmake Submodule 'deps/libuv' (https://github.com/JuliaLang/libuv.git) registered for path 'deps/libuv' Cloning into 'deps/libuv'... remote: Counting objects: 10053, done. remote: Compressing objects: 100% (2610/2610), done. remote: Total 10053 (delta 7830), reused 9534 (delta 7390) Receiving objects: 100% (10053/10053), 2.97 MiB | 376 KiB/s, done. Resolving deltas: 100% (7830/7830), done. Submodule path 'deps/libuv': checked out 'd48c5bd0adcb08fc8793529aed8a19530efde9ee' In file included from src/unix/eio/eio.c:51: src/unix/eio/ecb.h: In function 'ecb_unreachable': src/unix/eio/ecb.h:333: warning: 'noreturn' function does return % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 245k 100 245k 0 0 84477 0 0:00:02 0:00:02 --:--:-- 126k (Stripping trailing CRs from patch.) patching file dSFMT.h /usr/home/wsprague/julia/deps/install-name-FreeBSD.sh: not found gmake[2]: * [random/librandom.so] Error 127 gmake[1]: * [julia-release] Error 2 gmake: *\ [release] Error 2

[Keno]

can you try cp deps/install-name-Linux.sh deps/install-name-FreeBSD.sh?

[forkandwait]

Cranking away. Summary of changes: (1) patch to gpatch. (2) copied install-name. (3) changed the latter to /usr/local/bin/bash. (4) got rid of Microsoft Windows newlines in that file.

Thanks! I will update when it is finished building.

[forkandwait]

Whoopsie! Here is the next error.

In file included from ../common.h:326, from setparam_NANO.c:41: ../common_x86.h: In function 'blas_lock': ../common_x86.h:57: warning: implicit declaration of function 'sched_yield' setparam_NANO.c: In function 'get_l2_size_old': setparam_NANO.c:507: warning: unused variable 'cpuid_level' setparam_NANO.c: In function 'init_parameter': setparam_NANO.c:626: warning: unused variable 'l2' gmake[3]: wget: Command not found gmake[3]: * [lapack-3.4.1.tgz] Error 127 gmake[2]: * [openblas-v0.1.1/libopenblas.so] Error 2 gmake[1]: * [julia-release] Error 2 gmake: * [release] Error 2

[Keno]

This is an upstream issue. Just filed it here: https://github.com/xianyi/OpenBLAS/issues/106. For now, can you replace wget by curl in deps/openblas-v0.1.1/Makefile:262

[forkandwait]

Installed wget, restarted gmake, and got this error w/r/t lapack.

julia/ (j=2,r=0)$ gmake --2012-05-12 15:50:37-- http://www.netlib.org/lapack/lapack-3.4.1.tgz Resolving www.netlib.org (www.netlib.org)... 160.36.131.121 Connecting to www.netlib.org (www.netlib.org)|160.36.131.121|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 6147915 (5.9M) [application/x-gzip] Saving to: `lapack-3.4.1.tgz'

100%[===============================================================================================>] 6,147,915 378K/s in 16s

2012-05-12 15:50:54 (367 KB/s) - `lapack-3.4.1.tgz' saved [6147915/6147915]

md5sum: not found test: =: unexpected operator Cannot download lapack-3.4.1.tgz or the MD5 check sum is wrong (Please use orignal). gmake[3]: * [lapack-3.4.1] Error 1 gmake[2]: * [openblas-v0.1.1/libopenblas.so] Error 2 gmake[1]: * [julia-release] Error 2 gmake: * [release] Error 2

[Keno]

Seems like OpenBlas is trying to verify the md5 checksum of the downloaded package. Is there anyway to compute the MD5 checksum of a file on FreeBSD?

[forkandwait]

http://lists.freebsd.org/pipermail/freebsd-questions/2004-March/038150.html

Seems like "md5" is what we want. Never used it, so I don't know how it works exactly.

[forkandwait]

grepped and changed md5sum to md5 in the makefile, restarted make, got the following:

julia/ (j=2,r=0)$ gmake Cannot download lapack-3.4.1.tgz or the MD5 check sum is wrong (Please use orignal). gmake[3]: * [lapack-3.4.1] Error 1 gmake[2]: * [openblas-v0.1.1/libopenblas.so] Error 2 gmake[1]: * [julia-release] Error 2 gmake: * [release] Error 2

[Keno]

What does manually downloading http://www.netlib.org/lapack/lapack-3.4.1.tgz and using md5 lapack-3.4.1.tgz give you? If it gives you 44c3869c38c8335c2b9c2a8bb276eb55, try make -C deps distclean-openblas. If not, we'll have to find another way/ (maybe disable the MD5 check temporarily).

[forkandwait]

Yes, same md5. Ran your comand with "gmake", compiling now. will report.

Fails at some point with a missing "md5sum", probably because a Makefile gets downloaded after I go through and change it to only use "md5". I just symlinked md5sum to md5, and we will see if that makes it work.

[Keno]

This has been fixed upstream. Can you download https://github.com/xianyi/OpenBLAS/zipball/develop and extract it into the deps/openblas-v0.1.1 directory to verify that it works as expected?

[forkandwait]

I re-cloned and am building now. I crapped out with a bad md5sum, but I couldn't figure out where.

Any pointers on how to disable md5 checking, just in case?

[forkandwait]

I re-cloned, it failed on a bad md5sum. The check seems to be on line number 243 of openblas' Makefile -- all that awk stuff looks pretty tricky; it also might be relying on gnu awk, so I edited to "gawk". I downloaded the zip you specificied manually and it worked, but I didn't try to use it in the build. How do we disable md5sum checking, for now?

(Thanks for all your help! I might take a break for tonight soon, but I should be able to re-attack in the morning. Once we get it running, I will try to write a summary.)

double checked lapacks md5:

MD5 (lapack-3.4.1.tgz) = 44c3869c38c8335c2b9c2a8bb276eb55

Failed on this:

--2012-05-12 21:37:59--  http://www.netlib.org/lapack/lapack-3.4.1.tgz
Resolving www.netlib.org (www.netlib.org)... 160.36.131.121
Connecting to www.netlib.org (www.netlib.org)|160.36.131.121|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6147915 (5.9M) [application/x-gzip]
Saving to: `lapack-3.4.1.tgz'

100%[======================================>] 6,147,915    378K/s   in 17s     

2012-05-12 21:38:16 (362 KB/s) - `lapack-3.4.1.tgz' saved [6147915/6147915]

        Cannot download lapack-3.4.1.tgz or the MD5 check sum is wrong (Please use orignal).
gmake[3]: *** [lapack-3.4.1] Error 1
gmake[2]: *** [openblas-v0.1.1/libopenblas.so] Error 2
gmake[1]: *** [julia-release] Error 2
gmake: *** [release] Error 2

[pao: formatting]

[forkandwait]

I have gotten much farther with my freebsd build (will report later -- basically by using my PATH variable and a directory of symlinks to point to GNU tools). Now I am erroring out when it calls ld with gfortran. I just exported LD_RUN_PATH to include the symlink directory. I will report further when I have made more progress.

[StefanKarpinski]

Thanks for working on this. Would be great to officially support FreeBSD too!

[Keno]

This used to work a little while ago, but with nobody (that I known of) in the core development group using FreeBSD, it's really easy to lose track. Another reason we should get around to setting up some sort of CI sometime ;).

[forkandwait]

closer, but not finished (see below). Any ideas on how to get ld to find gfortran? gfortran46 is installed by FreeBSD ports, but I symlinked gfortran to it, and also set envars before compiling, to no avail. (Oh -- I am glad to work on -- FreeBSD awesome, and I desperately want a new math language to use ;))

END OF TESTS /usr/bin/ld: cannot find -lgfortran make[4]: * [../libopenblas-r0.1.1.so] Error 1 make[3]: * [shared] Error 2 make[2]: * [openblas-v0.1.1/libopenblas.so] Error 2 make[1]: * [julia-release] Error 2 make: *\ [release] Error 2 julia/ (j=0,r=2)$ which gfortran /home/wsprague/.GNUPATHS//gfortran

julia/ (j=0,r=0)$ echo $LD_LIBRARY_PATH /usr/local/bin/ julia/ (j=0,r=0)$ echo $LD_RUN_PATH
/usr/local/bin/

[pao]

It's looking for libgfortran, not the gfortran executable.

[forkandwait]

Should I symlink the lib?

Have to run, will try tonight, but I am still interested in any thoughts...

[forkandwait]

Created symlinks to libgfortran in /usr/lib (which is not a good thing to do on a FreeBSD machine). Was unable to figure out how to make export LDFLAGS+='-L/usr/local/lib/gcc46' actually work. It seemed like the makefiles couldn't see it, even though I marked it with export.

Now I have what looks like a bug:

/usr/bin/ld: cannot find -l/usr/local/lib make[2]: * [/usr/home/wsprague/julia/usr/lib/libsuitesparse_wrapper.so] Error 1 make[1]: * [julia-release] Error 2 make: *\ [release] Error 2

Any pointers on how to diagnose which Makefile could be fixed to deal with these problems?

Does julia really build on arbitrary Linux machines? I am sure it builds on the devs machines, but I would be comforted (and slightly surprised) to hear that it can be cloned and built on a random Ubuntu box.

This is hitting the point at which I don't really want to bother anymore, as I fail to rebuild dependencies that I know I have installed locally, etc, etc, etc. I think I am going to try one or two more times, but I might pitch it and come back in a few months...

[pao]

I don't know about "random Ubuntu box," but "typical Ubuntu box" works fine. Other devs use Fedora and OSX so we tend to notice when any of those break.

[forkandwait]

Oh well. I can't really spend any more time on it; I don't really know enough to know how to determine what the problems are at this point.

EDIT: I think I finally got the julia core to build, but failed on too many of the dependencies to make it worth continuing. Some of these dependencies are notoriously hard to build even on a Linux system, where they assume GNU tools, and when you try it on a FreeBSD system with multiple small differences (md5 != md5sum, make != gmake, etc, etc) it is a real pain, probably even for someone with better experience porting software than I have.

I also found the build process really hard to diagnose -- I crapped out with errors, but even with gmake -v, I couldn't figure out even what dependency package I was failing on.

On a FreeBSD system, it would probably be best to rely on pre-installed dependencies rather than building them, since these deps have been ported already with the ports system.

If I can help in any way, let me know, but I don't think I can work on this anymore, like I say above.

[pao]

For relying on system libraries, look in Make.inc for the USE_SYSTEM... flags.

[forkandwait]

Thanks for the pointer. I set everything to be 1, and made sure I installed it all, and then ran gmake, with the output below.

julia/ (j=0,r=127)$ gmake
SuiteSparse_wrapper.c:2:21: error: cholmod.h: No such file or directory
SuiteSparse_wrapper.c: In function 'jl_cholmod_common':
SuiteSparse_wrapper.c:7: error: 'cholmod_common' undeclared (first use in this function)
SuiteSparse_wrapper.c:7: error: (Each undeclared identifier is reported only once
SuiteSparse_wrapper.c:7: error: for each function it appears in.)
SuiteSparse_wrapper.c:7: error: 'c' undeclared (first use in this function)
SuiteSparse_wrapper.c:7: error: expected expression before ')' token
SuiteSparse_wrapper.c: In function 'jl_cholmod_dense':
SuiteSparse_wrapper.c:23: error: 'cholmod_dense' undeclared (first use in this function)
SuiteSparse_wrapper.c:23: error: 'mat' undeclared (first use in this function)
SuiteSparse_wrapper.c:23: error: expected expression before ')' token
SuiteSparse_wrapper.c: At top level:
SuiteSparse_wrapper.c:38: error: expected ')' before '*' token
SuiteSparse_wrapper.c: In function 'jl_cholmod_sparse':
SuiteSparse_wrapper.c:78: error: 'cholmod_sparse' undeclared (first use in this function)
SuiteSparse_wrapper.c:78: error: 's' undeclared (first use in this function)
SuiteSparse_wrapper.c:78: error: expected expression before ')' token
gmake[2]: *** [/usr/home/wsprague/julia/usr/lib/libsuitesparse_wrapper.so] Error 1
gmake[1]: *** [julia-release] Error 2
gmake: *** [release] Error 2

[pao: formatting]

[pao]

SuiteSparse is special; take a look at https://github.com/JuliaLang/julia/issues/785#issuecomment-5810242.

[forkandwait]

I set the flag in Make.inc to download and build suitesparse and it seems to get past that.

Now the problem is that nothing seems to be looking in /usr/local/(lib|include|whatever). When I symlink a library into /usr/lib then I make some progress, but have to symlink the next thing, etc...

EDIT: What i mean is "is there a place to set the search path for include and lib to include both /usr and /usr/local varieties, and have it propagate down? Like Make.inc?"

[nolta]

Julia should now build on FreeBSD 9.0, if you do the following:

1. Install packages

   # pkg_add -r gcc46 git gmake wget

2. Clone julia repository, and set FC = gfortran46 in Make.inc.

3. Start build

   $ cd /path/to/julia; gmake

4. Once it starts building openblas, Ctrl-C and apply the following patches:

   https://github.com/xianyi/OpenBLAS/commit/06e208c5c39911009f388275afe4097013d1abd2 https://github.com/nolta/OpenBLAS/commit/e9be1fdd2bf373800c1cf3a5217b09a018284b21

5. Restart build

   $ cd /path/to/julia; gmake test

6. Have margarita.

[ViralBShah]

@nolta that was some awesome work! I presume this can be closed once @forkandwait confirms.

[forkandwait]

@nolta thanks!

I will try tonight. I have a few questions though:

• How to I determine "when it starts building openblas"?
• Will these patches get incorporated upstream? I would either keep this ticket open or start a new one until the build is seamless.
• I am still a little skeptical that the build will be able to find my libraries and includes in /usr/local/*. Was this committed somewhere when I wasn't looking? ;)
• When I clone, I am getting MS DOS newlines in the files, even though I have git set up to correct to local newline types (or so I think). This screws up running the deps/install-blahblah-Freebsd.sh script. Thouhghts on how to fix?

All that being said... super thanks to all for working with me, Julia is is the most excited I have been about a language in a while, and everyone seems great!

[StefanKarpinski]

@forkandwait: you can force the download and compile of OpenBLAS immediately by doing make -C deps compile-openblas; kill it right after the download and untar of the source and then apply the patches. We are definitely in communication with the OpenBLAS guys and our upstream patches get incorporated pretty quickly, so those should be in the next point release, at which point we'll upgrade and get them too. After that just do a top-level make as usual and it will build everything else.

We definitely shouldn't have any files committed anywhere with MS DOS new lines. Which files are you seeing this for? I'll try to fix it.

[forkandwait]

I see newlines whenever I open any files in vi (emacs hides the conversions). little "^M" at the end of the lines. I might be misconfigured ("misconfigured from birth"... hehe)

[StefanKarpinski]

Somewhat ironically, the only file that contains the \r character anywhere in the source code is the Emacs mode contrib file. And I'm not entirely sure that's accidental. It maybe in there as a literal character in a regex expression.

[StefanKarpinski]

I'm not sure why that's happening, but I don't see any '\r' characters anywhere so it must be a misconfiguration on your end, no matter how bizarre that may seem.

[pao]

http://help.github.com/line-endings/ ?

[forkandwait]

Hmm. I am definitely seeing them on a new clone. I will try to figure it out -- maybe I am converting backwards.

[nolta]

Will these patches get incorporated upstream? I would either keep this ticket open or start a new one until the build is seamless.

Yes, these patches have been accepted upstream.

I am still a little skeptical that the build will be able to find my libraries and includes in /usr/local/*. Was this committed somewhere when I wasn't looking? ;)

My FreeBSD VM has the same uname -a as your system, so there's a decent chance this will work. That being said, i didn't have patch problems.

[forkandwait]

@nolta Could you run patch --version and maybe look to see if it is linked? Have you done any other linking (like of libraries)? You say VM -- did you download an image from somewhere (someone might have made the image "nicer" for Linux users)?

[nolta]

Could you run patch --version and maybe look to see if it is linked? Have you done any other linking (like of libraries)?

[nolta@freebsd90 ~]$ patch --version
Patch version 2.1
[nolta@freebsd90 ~]$ which patch
/usr/bin/patch

I'm guessing the patch problem is really just the ^M problem.

You say VM -- did you download an image from somewhere (someone might have made the image "nicer" for Linux users)?

Nope, i used the standard iso:

ftp://ftp.freebsd.org/pub/FreeBSD/releases/i386/i386/ISO-IMAGES/9.0/FreeBSD-9.0-RELEASE-i386-disc1.iso

and here's the list of packages:

[nolta@freebsd90 ~]$ pkg_info
bash-4.1.11         The GNU Project's Bourne Again SHell
binutils-2.22       GNU binary tools
ca_root_nss-3.12.11_1 The root certificate bundle from the Mozilla Project
curl-7.21.3_2       Non-interactive tool to get files from FTP, GOPHER, HTTP(S)
cvsps-2.1           Create patchset information from CVS
expat-2.0.1_2       XML 1.0 parser written in C
gcc-4.6.3.20111209  GNU Compiler Collection 4.6
gettext-0.18.1.1    GNU gettext package
git-1.7.8           Distributed source code management tool
gmake-3.82          GNU version of 'make' utility
gmp-5.0.2           A free library for arbitrary precision arithmetic
keychain-2.7.1      A user-friendly front-end to ssh-agent(1)
libiconv-1.13.1_1   A character set conversion library
libidn-1.22         Internationalized Domain Names command line tool
mpc-0.9             Library of complex numbers with arbitrarily high precision
mpfr-3.1.0_2        A library for multiple-precision floating-point computation
p5-Error-0.17016    Perl module to provide Error/exception support for perl: Er
p5-Net-SMTP-SSL-1.01_1 An SMTP client supporting SSL
perl-5.12.4_3       Practical Extraction and Report Language
pkg-config-0.25_1   A utility to retrieve information about installed libraries
python27-2.7.2_3    An interpreted object-oriented programming language
wget-1.13.4_1       Retrieve files from the Net via HTTP(S) and FTP

[forkandwait]

@nolta Well, if I actually read the blurb about newlines, and set git to "input", seems to work pretty well. I still needed to override using my gnu tools directory (to fix md5sum).

Hmm... I am currently bombing out when trying to connect to host for libunwind. So I have tried setting Make.inc to use it locally, but it still won't find my libs and includes (trying to compile again, so don't have the output to paste). Favor: could you try installing one of the dependencies (maybe readline?) and see if you get the same problem if you try to use it?

If we need to set LDFLAGS and CPPFLAGS to get us /usr/local/lib, can we do it in Make.inc? Maybe that is a bad approach -- I am just a humble scripter, so I am learning a lot about real projects and Makefiles and such....

FYI: I am using ports, not packages, and I have a lot of them ;)

Random thread about FreeBSD not checking /usr/local/* by default during compiles: http://forums.freebsd.org/showthread.php?t=19956

[nolta]

Are you still unable to download libunwind?

[forkandwait]

Woot! I got it to build!

md5 still doesn't work out of the box unless I tweak it like I said above. I wonder if it was binutils that gave you a linux style md5sum.

I am going to let @nolta close it if he thinks its ready. We should open a new ticket to create an official freebsd port (which might take care of some of the weird path stuff, we can set it to use gmake, etc). I will start helping on that next week, after I play with the language a little bit.

Thanks to everyone!

[StefanKarpinski]

Congrats to all involved in this effort. An official FreeBSD port would be incredible :-)

[nolta]

Excellent, i'm glad it works for you @forkandwait. I think i'll leave this open until the next version of openblas is released.

[forkandwait]

I think the only problem remaining is that my standard issue freebsd does not have md5sum, but rather md5. @nolta, could you run 'which md5sum' on your freebsd system? just curious, but maybe there could be a test in the Makefile. Also, I think md5sum (Linux) is called differently from md5 on FreeBSD.

[ViralBShah]

Doesn't md5 -r do the same thing as md5sum?

-viral

On 27-May-2012, at 10:10 AM, forkandwait wrote:

I think the only problem remaining is that my standard issue freebsd does not have md5sum, but rather md5. @nolta, could you run 'which md5sum' on your freebsd system? just curious, but maybe there could be a test in the Makefile. Also, I think md5sum (Linux) is called differently from md5 on FreeBSD.

---

Reply to this email directly or view it on GitHub: https://github.com/JuliaLang/julia/issues/829#issuecomment-5951457

[nolta]

I'll double check tomorrow, but I thought this was fixed by the openblas patches.