google-research / dex-lang

Research language for array processing in the Haskell/ML family
BSD 3-Clause "New" or "Revised" License
1.59k stars 107 forks source link

Trouble building on Mac #2

Closed oxinabox closed 4 years ago

oxinabox commented 5 years ago

I get part-way through the make set and I hit an error relating to hinotify-0.4

attoparsec          > [21 of 21] Compiling Data.Attoparsec
attoparsec          > copy/register
attoparsec          > Installing library in /Users/oxinabox/.stack/snapshots/x86_64-osx/19f92992da84ddd872e7f776ca44f
7ddcdce0f7b17815902723d4090d435093e/8.6.5/lib/x86_64-osx-ghc-8.6.5/attoparsec-0.13.2.2-6XZWdvg7dAKGuzy18iXIM3
attoparsec          > Registering library for attoparsec-0.13.2.2..

--  While building package hinotify-0.4 using:
      /Users/oxinabox/.stack/setup-exe-cache/x86_64-osx/Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.5 --builddir=.stack-wor
k/dist/x86_64-osx/Cabal-2.4.0.1 configure --user --package-db=clear --package-db=global --package-db=/Users/oxinabox/
.stack/snapshots/x86_64-osx/19f92992da84ddd872e7f776ca44f7ddcdce0f7b17815902723d4090d435093e/8.6.5/pkgdb --libdir=/Users/oxinabox/.stack/snapshots/x86_64-osx/19f92992da84ddd872e7f776ca44f7ddcdce0f7b17815902723d4090d435093e/8.6.5/lib -
-bindir=/Users/oxinabox/.stack/snapshots/x86_64-osx/19f92992da84ddd872e7f776ca44f7ddcdce0f7b17815902723d4090d435093e/
8.6.5/bin --datadir=/Users/oxinabox/.stack/snapshots/x86_64-osx/19f92992da84ddd872e7f776ca44f7ddcdce0f7b17815902723d4
090d435093e/8.6.5/share --libexecdir=/Users/oxinabox/.stack/snapshots/x86_64-osx/19f92992da84ddd872e7f776ca44f7ddcdce
0f7b17815902723d4090d435093e/8.6.5/libexec --sysconfdir=/Users/oxinabox/.stack/snapshots/x86_64-osx/19f92992da84ddd87
2e7f776ca44f7ddcdce0f7b17815902723d4090d435093e/8.6.5/etc --docdir=/Users/oxinabox/.stack/snapshots/x86_64-osx/19f929
92da84ddd872e7f776ca44f7ddcdce0f7b17815902723d4090d435093e/8.6.5/doc/hinotify-0.4 --htmldir=/Users/oxinabox/.stack/sn
apshots/x86_64-osx/19f92992da84ddd872e7f776ca44f7ddcdce0f7b17815902723d4090d435093e/8.6.5/doc/hinotify-0.4 --haddockd
ir=/Users/oxinabox/.stack/snapshots/x86_64-osx/19f92992da84ddd872e7f776ca44f7ddcdce0f7b17815902723d4090d435093e/8.6.5
/doc/hinotify-0.4 --dependency=async=async-2.2.2-EbxQ7tk0OFk9dJNMtaidSf --dependency=base=base-4.12.0.0 --dependency=
bytestring=bytestring-0.10.8.2 --dependency=containers=containers-0.6.0.1 --dependency=unix=unix-2.7.2.2 --exact-conf
iguration --ghc-option=-fhide-source-paths
    Process exited with code: ExitFailure 1
Progress 8/53
make: *** [all] Error 1

System Details:

~/ProgO/dex-lang(git:master)❱ ✘ ❱ llvm-config --version
7.1.0
~/ProgO/dex-lang(git:master)❱ ✔ ❱ stack --version
Version 2.1.3, Git revision 0fa51b9925decd937e4a993ad90cb686f88fa282 (7739 commits) x86_64 hpack-0.31.2
~/ProgO/dex-lang(git:master)❱ ✔ ❱
dougalm commented 5 years ago

First issue!

On non-Linux you can build without the web interface (which is the reason for the hinotify dep) using stack build --flag dex:-web.

I'll add another target to the makefile with a comment.

Closing, but please re-open if that didn't work for you. It would be nice to figure out the macOS equivalent to Linux's inotify. PRs welcome.

oxinabox commented 5 years ago

Well we got further this time 🙂

This time it was llvm-hs-7.0.1 that blocked me

JuicyPixels         > Registering library for JuicyPixels-3.3.3.1..

--  While building package llvm-hs-7.0.1 using:
      /private/var/folders/1r/4cjj1xk90qbb6njj7dn_cmt40000gn/T/stack2402/llvm-hs-7.0.1/.stack-work/dist/x86_64-osx/Cabal-2.4.0.1/setup/setup --builddir=.stack-work/dist/x86_64-osx/Cabal-2.4.0.1 configure --user --package-db=clear --package-db=global --package-db=/Users/oxinabox/.stack/snapshots/x86_64-osx/19f92992da84ddd872e7f776ca44f7ddcdce0f7b17815902723d4090d435093e/8.6.5/pkgdb --libdir=/Users/oxinabox/.stack/snapshots/x86_64-osx/19f92992da84ddd872e7f776ca44f7ddcdce0f7b17815902723d4090d435093e/8.6.5/lib --bindir=/Users/oxinabox/.stack/snapshots/x86_64-osx/19f92992da84ddd872e7f776ca44f7ddcdce0f7b17815902723d4090d435093e/8.6.5/bin --datadir=/Users/oxinabox/.stack/snapshots/x86_64-osx/19f92992da84ddd872e7f776ca44f7ddcdce0f7b17815902723d4090d435093e/8.6.5/share --libexecdir=/Users/oxinabox/.stack/snapshots/x86_64-osx/19f92992da84ddd872e7f776ca44f7ddcdce0f7b17815902723d4090d435093e/8.6.5/libexec --sysconfdir=/Users/oxinabox/.stack/snapshots/x86_64-osx/19f92992da84ddd872e7f776ca44f7ddcdce0f7b17815902723d4090d435093e/8.6.5/etc --docdir=/Users/oxinabox/.stack/snapshots/x86_64-osx/19f92992da84ddd872e7f776ca44f7ddcdce0f7b17815902723d4090d435093e/8.6.5/doc/llvm-hs-7.0.1 --htmldir=/Users/oxinabox/.stack/snapshots/x86_64-osx/19f92992da84ddd872e7f776ca44f7ddcdce0f7b17815902723d4090d435093e/8.6.5/doc/llvm-hs-7.0.1 --haddockdir=/Users/oxinabox/.stack/snapshots/x86_64-osx/19f92992da84ddd872e7f776ca44f7ddcdce0f7b17815902723d4090d435093e/8.6.5/doc/llvm-hs-7.0.1 --dependency=Cabal=Cabal-2.4.1.0-4t2ut7bCQNuEj8DDES6BZk --dependency=array=array-0.5.3.0 --dependency=attoparsec=attoparsec-0.13.2.2-6XZWdvg7dAKGuzy18iXIM3 --dependency=base=base-4.12.0.0 --dependency=bytestring=bytestring-0.10.8.2 --dependency=containers=containers-0.6.0.1 --dependency=exceptions=exceptions-0.10.2-2mS9GUVmJpmKSw5CuaN6ik --dependency=llvm-hs-pure=llvm-hs-pure-7.0.0-3a3CINtVYv5LBG1xtLCQFN --dependency=mtl=mtl-2.2.2 --dependency=template-haskell=template-haskell-2.14.0.0 --dependency=transformers=transformers-0.5.6.2 --dependency=utf8-string=utf8-string-1.0.1.1-Geq8jdOv4Q3LkcQoEOWDVv -f-debug -fshared-llvm --exact-configuration --ghc-option=-fhide-source-paths
    Process exited with code: ExitFailure 1
Progress 38/45

(I don't think I can reopen, but you can)

dougalm commented 5 years ago

I assume you have LLVM 7 installed on your system? (Stack won't do that for you.) I know @apaszke managed to build on Mac so it must be possible! I'll get my hands on a Mac and try to build it and report back if I learn anything.

sjakobi commented 5 years ago

I've also had trouble building llvm-hs, but on Ubuntu:

llvm-hs            > setup: The program 'llvm-config' version ==7.0.* is required but the version
llvm-hs            > found at /usr/bin/llvm-config is version 7.1.0

What's the best way to install llvm-7.0.* or to get around the version check?

oxinabox commented 5 years ago

I assume you have LLVM 7 installed on your system?

Yep, see above:

~/ProgO/dex-lang(git:master)❱ ✘ ❱ llvm-config --version
7.1.0

Installed via brew install llvm@7 then adding usr/local/opt/llvm@7/bin/ to my PATH

Hmm, it might be that I need to do something to add some libraries, e.g. with LD_PATH.

dougalm commented 5 years ago

@sjakobi , maybe try fiddling with the extra-deps in stack.yaml? I think I must have pinned those versions to match whatever LLVM version is on my system.

I'd welcome any suggestions for how to make building more robust. It's not something I'm very familiar with.

sjakobi commented 5 years ago

I've now installed llvm-8-dev via https://apt.llvm.org/, which appears to be compatible with llvm-hs-8.0.0. stack build and stack test now succeed with the following patch:

diff --git a/dex.cabal b/dex.cabal
index b397bc1..1be8656 100644
--- a/dex.cabal
+++ b/dex.cabal
@@ -21,7 +21,7 @@ library
                        Actor, DeShadow, Cat, Flops, Normalize,
                        Interpreter, RenderHtml, Plot
   build-depends:       base, containers, mtl, binary, bytestring,
-                       time, tf-random, llvm-hs-pure ==7.*, llvm-hs ==7.*,
+                       time, tf-random, llvm-hs-pure >= 7 && < 8.1, llvm-hs >= 7 && < 8.1,
                        aeson, megaparsec, warp, wai,
                        parser-combinators, http-types, prettyprinter, text,
                        data-binary-ieee754, blaze-html, cmark, diagrams-lib,
diff --git a/stack.yaml b/stack.yaml
index ebedbc7..878920f 100644
--- a/stack.yaml
+++ b/stack.yaml
@@ -10,5 +10,5 @@ packages:
 - .

 extra-deps:
-- llvm-hs-7.0.1
-- llvm-hs-pure-7.0.0
+- llvm-hs-8.0.0
+- llvm-hs-pure-8.0.0

However:

≻ stack exec -- dex repl
Pass 'Interprocedural Sparse Conditional Constant Propagation' is not initialized.
Verify if there is a pass dependency cycle.
Required Passes:
fish: “stack exec -- dex repl” terminated by signal SIGSEGV (Address boundary error)
dougalm commented 5 years ago

Eesh, I've never seen anything like that before. I've had a universally good experience with LLVM and llvm-hs so far. It's probably something on my end (I haven't tested with LLVM 8). I wonder what happens if you disable the LLVM passes by commenting out this line?

sjakobi commented 5 years ago

I wonder what happens if you disable the LLVM passes by commenting out this line?

Same result. :/

Sorry I can't be of more help. I don't know much about LLVM and it's my first time building llvm-hs.

joaogui1 commented 5 years ago

Now I'm suddenly having troubles too. When I make the project I get the following warning Warning: the following files would be used as linker inputs, but linking is not being done: cbits/libdex.so And then when I try to run dex I get: .../dex-lang/.stack-work/install/x86_64-linux/bf780b400f72a5a59241a8970e88cd31804e452b83a4a636831cdecd2d1c8c55/8.6.5/bin/dex: error while loading shared libraries: cbits/libdex.so: cannot open shared object file: No such file or directory but if I ls the cbits directory I do find libdex.so

dougalm commented 5 years ago

I thought I'd try updating everything to LLVM 8 but I can't get llvm-hs-8.0.0 to build ...

Stack does a really nice job of managing Haskell dependencies. So it's predictable that the build issues are around our single non-Haskell dependency.

Does anyone have ideas for how we could do this better? Building LLVM from source? Using a container?

joaogui1 commented 5 years ago

Building from source looks impractical for most people, maybe containers would be a good idea

dougalm commented 5 years ago

@joaogui1, I see that warning sometimes too and I don't think it's a problem. Try running dex from the project root directory (yes, it'd be nice if you didn't have to).

joaogui1 commented 5 years ago

@dougalm that worked, thanks :)

oxinabox commented 5 years ago

It would be nice to be on LLVM 8, I started trying to get this to build with the JuliaLang cross-compiler infastructure: BinaryBuilder. but @staticfloat informs me that that is setup for LLVM 8, and I would need to add a binary provider for LLVM 7, to use that. Which seems unfun.

If I can get BinaryBuilder going with this it would be nice, because then can just provide pre-built binaries for every platform; rather than needing to use make. (Not sure how much pain stack and haskell would give. Hopefully, can just use the stack installer script.)

joaogui1 commented 5 years ago

@joaogui1, I see that warning sometimes too and I don't think it's a problem. Try running dex from the project root directory (yes, it'd be nice if you didn't have to).

Wouldn't it be possible to add cbits/libdex.so to /usr/local/lib and then when installing dex make it reference /usr/local/lib/cbits/libdex.so instead of just cbits/libdex.so? Or create an enviroment variable with the path to libdex.so (when the person runs the makefile) and then tell dex to use that enviroment variable? The first option would require sudo and the second would be a little bit less stable I guess, what do you think?

dougalm commented 5 years ago

@oxinabox, that sounds promising! Thanks for offering to look into it.

It would be nice to be on LLVM 8

Yes, definitely. Let's move to LLVM 8 if we can solve the build issues.

then can just provide pre-built binaries for every platform; rather than needing to use make.

That sounds like the right long-term goal for sure. For now, though, the people wanting to build Dex will (hopefully) be people interested in helping develop it. So they'll need to be able to make changes to the Haskell source and recompile.

Dex might be different from Julia in this regard. AIUI, a lot of Julia is implemented in Julia. Dex is implemented in Haskell, even pieces like AD, and I don't see that changing. It's hard to imagine improving on Haskell as a language for writing compilers. There may eventually be a need for a standard library of numerical routines written in Dex. But even that is quite far off.

dougalm commented 5 years ago

@joaogui1, yes, both of those would work, but I prefer the second option because I don't think we need Dex to be available system-wide for now. I'd be happy to accept a PR for the readme that adds instructions for setting up the environment variable in your bashrc.

nottatdat commented 5 years ago

Same here on Mac. I have tried to:

All produced the same error.

oxinabox commented 5 years ago

Dex might be different from Julia in this regard. AIUI, a lot of Julia is implemented in Julia.

That shouldn't matter for purposes of BinaryBuilder, it is in theory a general purpose cross-compilation enviroment tool that happens to be maintained by the julia project. It is mostly used for building dependencies of julia and of julia packages. Like building GTK, and Tcl etc. I don't think anything based on Haskell yet has been built, so that might be a challange. But I'll give it a shot, should put the system through its paces.

For now, though, the people wanting to build Dex will (hopefully) be people interested in helping develop it. So they'll need to be able to make changes to the Haskell source and recompile.

Thats true. In this case it might help me, because it has containerization built-in so I don't need to worry about so much Mac OS weirdness (in theory).

But indeed, its isn't a real overall solution even if I can make it work.

oxinabox commented 5 years ago

Looks like my quest to get this to build with BinaryBuilder is blocked for now, because of https://github.com/commercialhaskell/stack/issues/2387

On a more positiive note: I now have DexLang working for myself. Because I gave up on Mac and just spun up a Ubuntu 16 VM.

apaszke commented 5 years ago

+1 for updating LLVM. I've actually never managed to build Dex on Mac exactly due to the LLVM version mismatch (7.1 installed, 7.0 expected). IIRC the LLVM website doesn't even have a packaged 7.0 version for Mac!

dougalm commented 5 years ago

@apaszke Ah, my bad. I forget that you (like Lyndon) failed even after disabling the inotify stuff. Ok, sounds like the right move is to switch to LLVM 8.

@oxinabox, glad you managed to get a VM version working at least! Also, I noticed you're at Invenia Cambridge. Say hi to Christian Steinruecken from me!

dougalm commented 5 years ago

Ok, I've updated to LLVM 8 on this branch, which includes a fix for @sjakobi's "not initialized" bug. I tried installing on a Mac, with LLVM 8 installed via brew install llvm@8, and now I'm stuck on this bug, which appears to be upstream of llvm-hs in LLVM itself. Or maybe homebrew LLVM doesn't work with llvm-hs?

I'll keep trying, but maybe someone here with more Mac familiarity can figure it out now that we're on LLVM 8?

dougalm commented 4 years ago

@oxinabox - did you have any luck building on Mac after we updated to LLVM 8?

oxinabox commented 4 years ago

I did not. Same error you were getting

I am currently building LLVM from source to see how that goes. Will post results on that.

oxinabox commented 4 years ago

The good news: I have dex running on Mac

The bad news. I can't repeat it. The first time I did it I kept a script so I would be able to repeat it. But apparently I missed something (or Dex master moved under me between my two runs).

My first shot seems to work fully But my second errors on start up of REPL

bash-3.2$ stack -- exec dex repl
dyld: lazy symbol binding failed: Symbol not found: _LLVMInitializeAMDGPUTargetInfo
  Referenced from: /Users/oxinabox/progmisc2/dex-lang/.stack-work/install/x86_64-osx/1051de0f5f44af8b4795ed2f3a6b75cc8f74343cb71df5fea9088f8e9b40d75d/8.6.5/bin/dex
  Expected in: /usr/local/lib/libLLVM.dylib

dyld: Symbol not found: _LLVMInitializeAMDGPUTargetInfo
  Referenced from: /Users/oxinabox/progmisc2/dex-lang/.stack-work/install/x86_64-osx/1051de0f5f44af8b4795ed2f3a6b75cc8f74343cb71df5fea9088f8e9b40d75d/8.6.5/bin/dex
  Expected in: /usr/local/lib/libLLVM.dylib

Abort trap: 6

I am just checking over my script on how I built it to make sure that that is right -- I might have missed a step the second time

dougalm commented 4 years ago

Hooray, that's great news! Thank you for working on this. Sounds like we're very close to a reproducible Mac build.

oxinabox commented 4 years ago

How to build and run dex on Mac.

To install Dex

# Set envvars as desired
export PROJECT=/Users/oxinabox/dex_building/round6
export LLVM_DIR="$PROJECT"/llvm/
export LLVM_INSTALL="$LLVM_DIR"/installed
export DEX_SRC="$PROJECT"/dex-lang

#### Build LLVM ####
# as per https://github.com/llvm-hs/llvm-hs/tree/llvm-9#building-from-source

# Download LLVM 
mkdir -p $LLVM_DIR
cd $LLVM_DIR
wget http://releases.llvm.org/9.0.0/llvm-9.0.0.src.tar.xz
tar -xzf llvm-9.0.0.src.tar.xz

export LLVM_SRC=$LLVM_DIR/llvm-9.0.0.src/

# Build LLVM (this will take a while)
mkdir builddir
cd builddir

cmake $LLVM_SRC -DCMAKE_INSTALL_PREFIX=$LLVM_INSTALL  \
-DLLVM_BUILD_LLVM_DYLIB=True -DLLVM_LINK_LLVM_DYLIB=True

cmake --build . -j 4
cmake --build . --target install 

# Special Mac things
cd $LLVM_INSTALL/lib
ln -s -F libLLVM.dylib libLLVM-9.dylib
install_name_tool -id $PWD/libLTO.dylib libLTO.dylib
install_name_tool -id $PWD/libLLVM.dylib libLLVM.dylib
install_name_tool -change '@rpath/libLLVM.dylib' $PWD/libLLVM.dylib libLTO.dylib

## Now set this as your LLVM
export PATH="$LLVM_INSTALL"/bin:$PATH
llvm-config --libdir # Check this matches to `$"LLVM_INSTALL"/lib 

#### Build Dex
mkdir -p $DEX_SRC
git clone https://github.com/google-research/dex-lang.git $DEX_SRC
cd $DEX_SRC
make -j4

Building LLVM takes literally hours. So not great.

To run Dex

(update: LD_LIBRARY_PATH does not need to be set, I made some other mistake, see below)

# None of the above need to be set except to set `LD_LIBRARY_PATH`
export LD_LIBRARY_PATH=$"$LLVM_INSTALL"/lib:$LD_LIBRARY_PATH
stack -- exec dex repl

I would really like to get rid of the need to set the LD_LIBRARY_PATH. If you don't do that then it gets the dyld: lazy symbol binding failed when you start it. Something needs to be changes somewhere so dex uses statically linked libraries I think? I tried running llvm-config --link-static before building dex, but that did not seem to work. I am no expert on LLVM build/linking stuff.

I know its possible, because I still have the dex build I made yesterday that works fine without setting the LD_LIBRARY_PATH. I just can't recreate it.

oxinabox commented 4 years ago

Ok, so what is going on is that the path for the custom build of LLVM is not getting put in to the dex binary. Which right now is built at /Users/oxinabox/progmisc2/dex-lang/.stack-work/install/x86_64-osx/1051de0f5f44af8b4795ed2f3a6b75cc8f74343cb71df5fea9088f8e9b40d75d/8.6.5/bin

looking at the good one I built yesterday, under ❱ otool -l -v dex | less it has:

name /Users/oxinabox/progmisc/llvm-8.0.0.installed/lib/libLLVM.dylib (offset 24)

Where as todays has:

name @rpath/libLLVM.dylib (offset 24)

And no LC_RPATH defined. So I guess that is why it goes looking in system shared library locations.

If I go there and run install_name_tool -add_rpath "$LLVM_INSTALL"/lib dex to add a LC_RPATH then that makes it work.

oxinabox commented 4 years ago

Ok, after much fiddling, and learning things I never wanted to know about linking :joy: I have concluded my original instructions were infact correct. And I just messed up the

# Special Mac things
cd $LLVM_INSTALL/lib
ln -s libLLVM.dylib libLLVM-8.dylib
install_name_tool -id $PWD/libLTO.dylib libLTO.dylib
install_name_tool -id $PWD/libLLVM.dylib libLLVM.dylib
install_name_tool -change '@rpath/libLLVM.dylib' $PWD/libLLVM.dylib libLTO.dylib

step.

Once can verify that that step that was done correctly via: otool -D You should get:

~/progm/llvm-/lib❱ ✔ ❱ pwd
/Users/oxinabox/progmisc2/llvm-8.0.0.installed/lib
~/progm/llvm-/lib❱ ✔ ❱ otool -D libLLVM.dylib
libLLVM.dylib:
/Users/oxinabox/progmisc2/llvm-8.0.0.installed/lib/libLLVM.dylib

Rather than:

~/progm/backu/lib❱ ✔ ❱ otool -D libLLVM.dylib
libLLVM.dylib:
@rpath/libLLVM.dylib
dougalm commented 4 years ago

Wow, heroic work! I'll link to your "how-to" message in the README. Thanks for digging into this!

oxinabox commented 4 years ago

I am trying to see if I can workout a way to do it with some prebuilt LLVM libraries because building LLVM takes ages

andrewnc commented 4 years ago

Excellent work on the tutorial! Unfortunately, I don't have the 15+GB disk space on my mac to build LLVM. I look forward to prebuilt libraries etc.

oxinabox commented 4 years ago

I have given bup on building and distributing LLVM libraries for now, after making no progress for a while. Mac OS Catelina is also increasing hostile to distributing ... anything with the latest increases requiring you to send them any binaries to sign.

You might want to buy a 32GB USB stick or something.

oxinabox commented 4 years ago

https://github.com/google-research/dex-lang/pull/61 removes the need for Stack, which was issues with musl, and thus with the BinaryBuilder cross compilation infastructure i was working with before. This means I might be able to get it working (if i can find time) so we can have be built binaries for Mac.

apaszke commented 4 years ago

@oxinabox awesome! Please keep in mind that it does require Cabal 3, but I think that's the default you get from Homebrew

oxinabox commented 4 years ago

I have updated the instructions above so it now works with LLVM v9

lancelet commented 4 years ago

I found that I was able to use the llvm@9 libraries from Homebrew if I passed the Cabal flag to llvm-hs to use static linking. Given that a makefile is used, the simplest way seemed to be to add the following to stack.yaml:

flags:
  llvm-hs:
    shared-llvm: false

That flag is described in the llvm-hs documentation.

It also requires compiling with /usr/local/opt/llvm@9/bin on the PATH, so that llvm-hs can find the llvm-config binary. (There's a note about why Homebrew doesn't add this path by default when llvm@9 is installed; it comes down to the fact that they want to allow multiple LLVM versions to be installed simultaneously.)

Apparently, linking statically against LLVM is not the "preferred" approach, but there's a Homebrew issue (and again here) where the Homebrew maintainers are refusing a patch to fix the dylib problem we're encountering, and saying it can only be addressed upstream. While that all gets sorted out, static linking seems like a solution with minimal fuss.

apaszke commented 4 years ago

@lancelet that is awesome! Can you please try wrapping this path in if os(darwin) and adding that to our Cabal file if that works?

lancelet commented 4 years ago

@apaszke sure, I'll take a look to see if that's possible. It might take me a few days to get to it though.

apaszke commented 4 years ago

Of course, take your time! Thanks for helping us figure it out

lancelet commented 4 years ago

I'm not sure that there's any cabal-native way to add this path when building.

The core problem is that llvm-config is a system binary required by llvm-hs (link to its cabal file), rather than by Dex itself. So while there might (perhaps) be ways to hack in the additional path within the llvm-hs cabal file, AFAICT, it's well outside the scope of the Dex cabal file to specify what path llvm-hs might use when building.

I've read through as much of the Cabal docs as I can on this, but there might be something I've missed. I haven't come across anything that seems as though it would provide this facility, however.

8bitmp3 commented 4 years ago

https://github.com/google-research/dex-lang/issues/2#issuecomment-557793009 <- That's a good writeup @oxinabox Thanks very much. I managed to get as far as step 136/138 by following your guide after running make -j4 to build Dex:

.../.stack/setup-exe-cache/x86_64-osx/Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.5 --builddir=.stack-work/dist/x86_64-osx/Cabal-2.4.0.1 build --ghc-options " -fdiagnostics-color=always"
    Process exited with code: ExitFailure 1
Progress 136/138
make: *** [build] Error 1
apaszke commented 4 years ago

@8bitmp3 can you please post some more context of the failure?

8bitmp3 commented 4 years ago

Hey @apaszke , yes sure. These are the steps I took.

  1. Install stack with curl -sSL https://get.haskellstack.org/ | sh

  2. Install LLVM 9 as follows:

# 1. Create a directory for LLVM 9
mkdir llvm
cd llvm

# 2. Download and unpack LLVM 9.0.0
curl -O https://releases.llvm.org/9.0.0/llvm-9.0.0.src.tar.xz
tar -xzf llvm-9.0.0.src.tar.xz

# 3. Set up a build directory for LLVM
mkdir builddir

# 4. Set up separate source and installed directories 
mkdir llvm-9.0.0.src
mkdir installed

# 5. Create useful environment variables
export LLVM_DIR="/Users/{USER}/llvm"
export LLVM_INSTALL="$LLVM_DIR"/installed
export LLVM_SRC=$LLVM_DIR/llvm-9.0.0.src/

# 6. Build/configure LLVM
cd builddir
cmake $LLVM_SRC -DCMAKE_INSTALL_PREFIX=$LLVM_INSTALL  -DLLVM_BUILD_LLVM_DYLIB=True -DLLVM_LINK_LLVM_DYLIB=True
cmake --build . -j 4
cmake --build . --target install

cd $LLVM_INSTALL/lib
ln -s -F libLLVM.dylib libLLVM-9.dylib
install_name_tool -id $PWD/libLTO.dylib libLTO.dylib
install_name_tool -id $PWD/libLLVM.dylib libLLVM.dylib
install_name_tool -change '@rpath/libLLVM.dylib' $PWD/libLLVM.dylib libLTO.dylib

export PATH="$LLVM_INSTALL"/bin:$PATH
llvm-config --libdir

# 8. Install Dex
mkdir dex-source
cd dex-source
export DEX_SRC="/Users/{USER}/dex-source"
git clone https://github.com/google-research/dex-lang.git
cd dex-lang
make -j4

After running the last command—make -j4—the output indicated that there were 138 "steps" (?) and all was going well until it hit 136/138:

.../.stack/setup-exe-cache/x86_64-osx/Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.5 --builddir=.stack-work/dist/x86_64-osx/Cabal-2.4.0.1 build --ghc-options " -fdiagnostics-color=always"
    Process exited with code: ExitFailure 1
Progress 136/138
make: *** [build] Error 1

cc @oxinabox

jobovy commented 4 years ago

I just tried compiling dex on a Mac and can report that just installing stack, llvm@9 using Homebrew (and making sure that it's on the PATH and in CFLAGS/LDFLAGS), and editing the stack.yml file as stated in this comment worked perfectly!

apaszke commented 4 years ago

@8bitmp3 please make sure that llvm-config (from llvm@9) is on your PATH. Otherwise building the llvm-hs package that we depend on will fail.

oxinabox commented 4 years ago

This is the llvm-config --libdir # Check this matches to $"LLVM_INSTALL"/lib step. If I was better at shell scripting that would be a assertion

gilbo commented 4 years ago

I was having some trouble with OSX installation. The following 2 fixes got me through to a successful make tests

1) I ran into problems with C99 and aligned_alloc. I simply replaced line 18 of cbits/libdex.c, i.e.

  char* result = aligned_alloc(alignment, nbytes);

with the following alternative POSIX alternative call

  char* result = NULL;
  if(posix_memalign(&result, alignment, nbytes) != 0)
    result = NULL; // make sure errors are correctly reported

2) I followed the instructions on the llvm-hs project page for how to install on OS X. They recommended a specific homebrew-based installation of LLVM that their project appears set up to detect. (command brew install llvm-hs/llvm/llvm-9 at this time) Maybe directing Dex users to their installation instructions (rather than this thread?) might be more reliable?