OpenBSD installer / instructions

commercialhaskell / stack

The Haskell Tool Stack

http://haskellstack.org

BSD 3-Clause "New" or "Revised" License

3.98k stars 842 forks source link

OpenBSD installer / instructions #416

Closed mrijkeboer closed 7 years ago

mrijkeboer commented 9 years ago

Any chance of getting a binary installer for OpenBSD (current, AMD64) or instructions on how to get it working from source? I develop Haskell on OpenBSD and would like to use stack.

borsboom commented 8 years ago

Steps remaining for official OpenBSD support:

[x] Have stack setup use bzcat/xzcat to uncompress GHC bindist instead of relying on tar to do it (see also https://github.com/commercialhaskell/stack/issues/2283)
[x] Await a Stackage LTS snapshot that uses GHC 8.0.x and update the Stack build process (and integration tests) to use it
[ ] Ensure integration tests pass
[ ] Integrate into release process (I've already added a starter Vagrantfile in etc/vagrant/openbsd-5.9-amd64/Vagrantfile)
[x] Perhaps remove old not-quite-working OpenBSD bindists from stack-setup-2.yaml, to avoid confusion.

Blaisorblade commented 8 years ago

https://github.com/commercialhaskell/stack/pull/2456#issuecomment-238330797 confirms that ghc-7.8.3.20150623 fails installation, though there was no actual report here AFAICS. Opening the tarball reveals the extracted folder is named ghc-7.8.3.20150623 rather than ghc-7.8.3 as expected in the code.

seanwestfall commented 7 years ago

Hey guys, is anyone still working on this. Recently tried a cabal install stack on OpenBSD 6.0 on an AMD64 virtual machine.

Got this error during compiling:

# cabal install stack
...
cabal: user error (Error: some packages failed to install:
stack-1.2.0 depends on store-0.2.1.2 which failed to install.
store-0.2.1.2 failed during the building phase. The exception was:
ExitFailure (-10)
)
# cabal install store-0.2.1.2
Resolving dependencies...
Configuring store-0.2.1.2...
Building store-0.2.1.2...
Preprocessing library store-0.2.1.2...
[ 1 of 10] Compiling System.IO.ByteBuffer ( src/System/IO/ByteBuffer.hs, dist/build/System/IO/ByteBuffer.o )
[ 2 of 10] Compiling Data.Store.Impl  ( src/Data/Store/Impl.hs, dist/build/Data/Store/Impl.o )
[ 3 of 10] Compiling Data.Store.TH    ( src/Data/Store/TH.hs, dist/build/Data/Store/TH.o )
[ 4 of 10] Compiling Data.Store.TH.Internal ( src/Data/Store/TH/Internal.hs, dist/build/Data/Store/TH/Internal.o )
[ 5 of 10] Compiling Data.Store.Internal ( src/Data/Store/Internal.hs, dist/build/Data/Store/Internal.o )
Failed to install store-0.2.1.2
cabal: user error (Error: some packages failed to install:
store-0.2.1.2 failed during the building phase. The exception was:
ExitFailure (-10)
)
#

So obviously this is a store-0.2.1.2 issue, I opened an issue there as well: https://github.com/fpco/store/issues/85

seanwestfall commented 7 years ago

hey guys, fixed the issue by doubling the ram on the virtual machine. So ignore the comment above.

dhouthoo commented 7 years ago

Hi all,

with ghc 8.0.1 and ghc 8.0.2 making a ghc bindist works pretty much out-of-the-box on OpenBSD 6.0 x86_64. I used my build (8.0.1) to compile cabal, stack and the default stack project template without a problem.

Karel Gardas, who provides the bindist on the ghc site, uses OpenBSD current to build. So that's why those bindist are not working on 6.0. Anyway, using the ./configure options he gave me, the ghc build on 6.0 just works:

pkg_add ghc gmake autoconf automake iconv gmp libffi ./configure \ --with-iconv-libraries=/usr/local/lib \ --with-iconv-includes=/usr/local/include \ --with-gmp-libraries=/usr/local/lib \ --with-gmp-includes=/usr/local/include \ --with-ffi-libraries=/usr/local/lib \ --with-ffi-includes=/usr/local/include \ --with-sytem-libffi gmake gmake binary-dist

What's the way forward to unblock this?

Blaisorblade commented 7 years ago

Karel Gardas, who provides the bindist on the ghc site, uses OpenBSD current to build. So that's why those bindist are not working on 6.0.

I'm confused? GHC 8.0.1/8.0.2 bindists are documented as targeted for OpenBSD 5.9 (and that was confirmed for GHC 8.0.1 in earlier posts) — is that inaccurate? And IIUC OpenBSD current is later than 6.0 and stuff built there won't work on older releases? Finally: would binaries built on earlier releases work on later ones, or viceversa?

https://www.haskell.org/ghc/download_ghc_8_0_1#openbsd_x86_64 https://www.haskell.org/ghc/download_ghc_8_0_2#openbsd_x86_64

Anyway: supporting more variants of a platform is just a matter of listing those correctly in https://github.com/fpco/stackage-content/blob/master/stack/stack-setup-2.yaml (see all the Windows variants there to wit). But keep in mind that by default Stack uses externally maintained bindists (by default, official ones—see https://github.com/commercialhaskell/stack/issues/416#issuecomment-220405106).

What's the way forward to unblock this?

This is relevant: https://github.com/commercialhaskell/stack/issues/416#issuecomment-220801230 Maybe @borsboom has news?

Of those, we have an LTS based on GHC 8.0.1, and Stack can build with it; it seems integration tests still don't default to that version though.

It seems the Vagrantfile mentioned by that comment is still there (based on OpenBSD 6.0). It would be good to update it. Not sure what's needed since I don't take care of releases (IIRC @borsboom does), but I'd guess we need an uploaded bindist (mentioned in https://github.com/fpco/stackage-content/blob/master/stack/stack-setup-2.yaml) and a Vagrantfile to install it and test it.

Blaisorblade commented 7 years ago

After seeing #2822, I investigated myself.

TL;DR. Indeed, GHC bindists, or userland programs, in general, aren't necessarily portable from OpenBSD version X to OpenBSD version Y. It seems one would need a different bindist for each supported OpenBSD release.

Sources: Based on https://www.openbsd.org/faq/faq5.html#Flavors and http://comp.unix.bsd.openbsd.misc.narkive.com/ey6FWylS/backward-binary-compatibility#post4 ages ago (yet the second relevant google result for openbsd binary compatibility),, it seems binary compatibility with a release is maintained on the -stable channel.

The latter link mentioned static linking as an option. Would that still work?

mrijkeboer commented 7 years ago

@Blaisorblade I'm not sure about GHC itself, but if I statically compile the very simple Haskell program below on OpenBSD-current it won't run on OpenBSD 6.0-stable.

On OpenBSD-current:

$ sysctl kern.version
kern.version=OpenBSD 6.0-current (GENERIC.MP) #158: Mon Jan 30 19:30:12 MST 2017
    deraadt@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP

$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 7.10.3

$ cat Main.hs                                                                 
main :: IO ()
main = putStrLn "Hello"

$ ghc -optl-static Main.hs
Linking Main ...
/usr/local/lib/ghc/rts/libHSrts.a(RtsFlags.o): In function `copyArg':

rts/RtsFlags.c:1685:0:
     warning: warning: strcpy() is almost always misused, please use strlcpy()
/usr/local/lib/ghc/rts/libHSrts.a(RtsUtils.o): In function `showStgWord64':

rts/RtsUtils.c:204:0:
     warning: warning: sprintf() is often misused, please use snprintf()

$ file Main
Main: ELF 64-bit LSB executable, x86-64, version 1

$ ldd Main
Main:
not a dynamic executable

$ ./Main 
Hello

$ sha256 Main                                                                   
SHA256 (Main) = 7ad681ca78976e5401b548afc768a3d4d1137745ee18ea71b5e6688d36f0a144

When I copy the executable to OpenBSD 6.0-stable it get the following:

$ sysctl kern.version
kern.version=OpenBSD 6.0-stable (GENERIC.MP) #8: Thu Jan  5 19:41:14 CET 2017
    build@dexter.bunix.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP

$ sha256 Main                                                                   
SHA256 (Main) = 7ad681ca78976e5401b548afc768a3d4d1137745ee18ea71b5e6688d36f0a144

$ ./Main
./Main[1]: syntax error: `H$IE�L$I�H$C��^Gu^X$?#ff$Pff$P' unexpected

However if I build the same program on OpenBSD 6.0-stable and copy it to OpenBSD-current it works, although I'm not sure if this is by design or just luck.

On OpenBSD 6.0-stable:

$ sysctl kern.version
kern.version=OpenBSD 6.0-stable (GENERIC.MP) #7: Sat Nov  5 21:29:21 CET 2016
    build@dexter.bunix.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP

$ cat Main.hs                                                                 
main :: IO ()
main = putStrLn "Hello"

$ ghc -optl-static Main.hs  
Linking Main ...
/usr/local/lib/ghc/rts/libHSrts.a(RtsFlags.o): In function `copyArg':

rts/RtsFlags.c:1685:0:
     warning: warning: strcpy() is almost always misused, please use strlcpy()
/usr/local/lib/ghc/rts/libHSrts.a(RtsUtils.o): In function `showStgWord64':

rts/RtsUtils.c:204:0:
     warning: warning: sprintf() is often misused, please use snprintf()

$ file Main 
Main: ELF 64-bit LSB executable, x86-64, version 1

$ ldd Main
Main:
not a dynamic executable

$ ./Main                                                                      
Hello

$ sha256 Main 
SHA256 (Main) = afece87c07c79b68737efca66bdde280d84202cec4f33bd34fbc3b6a1431abc9

On OpenBSD-current:

$ sysctl kern.version
kern.version=OpenBSD 6.0-current (GENERIC.MP) #158: Mon Jan 30 19:30:12 MST 2017
    deraadt@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP

$ sha256 Main                       
SHA256 (Main) = afece87c07c79b68737efca66bdde280d84202cec4f33bd34fbc3b6a1431abc9

$ ./Main                                                                      
Hello

Blaisorblade commented 7 years ago

@mrijkeboer Thanks for trying this out! The more interesting direction is building on an older version and trying it out on a new one—the reverse cannot work by design, if -current adds new syscalls and uses them in libc. Since I fail to find clear info for this use case, could some OpenBSD user interested ask in the OpenBSD community (or the Haskell OpenBSD community)? I'm not clueful enough on OpenBSD to ask on the OpenBSD mailing list.

Sticking to system GHCs is of course an alternative, especially if ports allow building older GHCs. The main downside is that Stack won't be able to pick the appropriate version, but that could be solved if somebody picked up #2433, which is more generally useful (maybe see also #2546).

mrijkeboer commented 7 years ago

@Blaisorblade It seems that the GHC 8.0.2 is build for OpenBSD-current or at least it works:

$ sysctl kern.version
kern.version=OpenBSD 6.0-current (GENERIC.MP) #166: Wed Feb  8 19:15:03 MST 2017
    deraadt@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP

$ stack --version
Version 1.3.2 x86_64 hpack-0.15.0

$ stack new test82
Downloading template "new-template" to create project "test82" in test82/ ...

The following parameters were needed by the template but not provided: category
You can provide them in /home/user/.stack/config.yaml, like this:
templates:
  params:
    category: value
Or you can pass each one as parameters like this:
stack new test82 new-template -p "category:value"

Looking for .cabal or package.yaml files to use to init the project.
Using cabal packages:
- test82/test82.cabal

Selecting the best among 10 snapshots...

Downloaded lts-8.0 build plan.    
Updating package index Hackage (mirrored at https://github.com/commercialhaskell                                                                                Fetching package index ...remote: Counting objects: 3689, done.
remote: Compressing objects: 100% (246/246), done.
remote: Total 3689 (delta 1313), reused 1262 (delta 1262), pack-reused 2175
Receiving objects: 100% (3689/3689), 2.83 MiB | 725.00 KiB/s, done.
Resolving deltas: 100% (1490/1490), completed with 556 local objects.
From https://github.com/commercialhaskell/all-cabal-hashes
   79f894d3f9..53fcf98366  hackage         -> origin/hackage
 t [tag update]            current-hackage -> current-hackage
Fetched package index.    
Populated index cache.    
* Matches lts-8.0

Selected resolver: lts-8.0
Initialising configuration using resolver: lts-8.0
Total number of user packages considered: 1
Writing configuration to file: test82/stack.yaml
All done.

$ cd test82/

$ stack setup
Preparing to install GHC to an isolated location.
This will not interfere with any system-level installation.
Downloaded ghc-8.0.2.                                     
Unpacking GHC into /home/user/.stack/programs/x86_64-openbsd/ghc-8.0.2.temp/                                                                                 Installed GHC.     
stack will use a sandboxed GHC it installed
For more information on paths, see 'stack path' and 'stack exec env'
To use this GHC and packages outside of a project, consider using:
stack ghc, stack ghci, stack runghc, or stack exec

$ stack build
Warning: File listed in test82.cabal file does not exist: README.md
[1 of 2] Compiling Main             ( /home/user/.stack/setup-exe-src/setup-mPHDZzAJ.hs, /home/user/.stack/setup-exe-src/setup-mPHDZzAJ.o )
[2 of 2] Compiling StackSetupShim   ( /home/user/.stack/setup-exe-src/setup-shim-mPHDZzAJ.hs, /home/user/.stack/setup-exe-src/setup-shim-mPHDZzAJ.o )
Linking /home/user/.stack/setup-exe-cache/x86_64-openbsd/tmp-Cabal-simple_mPHDZzAJ_1.24.2.0_ghc-8.0.2 ...
/home/user/.stack/programs/x86_64-openbsd/ghc-8.0.2/lib/ghc-8.0.2/base-4.9.1.0/libHSbase-4.9.1.0.a(IO__1.o): In function `ghczuwrapperZC0ZCbaseZCSystemziIOZCrand':
(.text+0x1): warning: warning: rand() may return deterministic values, is that what you want?
/home/user/.stack/programs/x86_64-openbsd/ghc-8.0.2/lib/ghc-8.0.2/rts/libHSrts_thr.a(RtsFlags.thr_o): In function `copyArg':

rts/RtsFlags.c:1805:0: error:
     warning: warning: strcpy() is almost always misused, please use strlcpy()
/usr/local/lib/libgmp.so.10.0: warning: warning: vsprintf() is often misused, please use vsnprintf()
/home/user/.stack/programs/x86_64-openbsd/ghc-8.0.2/lib/ghc-8.0.2/rts/libHSrts_thr.a(RtsUtils.thr_o): In function `showStgWord64':

rts/RtsUtils.c:230:0: error:
     warning: warning: sprintf() is often misused, please use snprintf()
test82-0.1.0.0: configure (lib + exe)
Configuring test82-0.1.0.0...
test82-0.1.0.0: build (lib + exe)
Preprocessing library test82-0.1.0.0...
[1 of 1] Compiling Lib              ( src/Lib.hs, .stack-work/dist/x86_64-openbsd/Cabal-1.24.2.0/build/Lib.o )
Preprocessing executable 'test82-exe' for test82-0.1.0.0...
[1 of 1] Compiling Main             ( app/Main.hs, .stack-work/dist/x86_64-openbsd/Cabal-1.24.2.0/build/test82-exe/test82-exe-tmp/Main.o )
Linking .stack-work/dist/x86_64-openbsd/Cabal-1.24.2.0/build/test82-exe/test82-exe ...
/home/user/.stack/programs/x86_64-openbsd/ghc-8.0.2/lib/ghc-8.0.2/rts/libHSrts_thr.a(RtsFlags.thr_o): In function `copyArg':

rts/RtsFlags.c:1805:0: error:
     warning: warning: strcpy() is almost always misused, please use strlcpy()
/usr/local/lib/libgmp.so.10.0: warning: warning: vsprintf() is often misused, please use vsnprintf()
/home/user/.stack/programs/x86_64-openbsd/ghc-8.0.2/lib/ghc-8.0.2/rts/libHSrts_thr.a(RtsUtils.thr_o): In function `showStgWord64':

rts/RtsUtils.c:230:0: error:
     warning: warning: sprintf() is often misused, please use snprintf()
Warning: File listed in test82.cabal file does not exist: README.md
test82-0.1.0.0: copy/register
Installing library in
/home/user/Scratch/test82/.stack-work/install/x86_64-openbsd/lts-8.0/8.0.2/lib/x86_64-openbsd-ghc-8.0.2/test82-0.1.0.0-EVLQkMVlEaDLN2y26yqY67
Installing executable(s) in
/home/user/Scratch/test82/.stack-work/install/x86_64-openbsd/lts-8.0/8.0.2/bin
Registering test82-0.1.0.0...

$ stack exec test82-exe
someFunc

mrijkeboer commented 7 years ago

@Blaisorblade I've asked the OpenBSD mailing list whether static binaries compiled on an older release (X) are expected to run on a newer release (X+1, X+2) [0]. I've received the following - expected - answer from Sebastien Marie [1]:

The generic answer will be "no".

The example is the switch from 5.4 to 5.5 release which included time_t change (32 to 64 bits) - see https://www.openbsd.org/faq/upgrade55.html#time_t

But generally, an old binary (from release X) is able to run on a new kernel (from release X+1), but nothing more could be expected: old things are cleaned, so an old binary could be able to run or not (it just depends if relying on old API/ABI with kernel - syscalls, struct size...).

So it seems that the best way would be to support more variants of the OpenBSD platform like you suggested above. Fortunately OpenBSD only maintains three releases at a time:

Current development version (-current).
Latest release (at the time of this writing: 6.0).
Previous release (at the time of this writing: 5.9).

As Juan Francisco Cantero Hurtado [2] pointed out, Go on OpenBSD has the same problem: https://github.com/golang/go/issues/15227

[0] http://marc.info/?l=openbsd-misc&m=148793092509010&w=2 [1] http://marc.info/?l=openbsd-misc&m=148793453010027&w=2 [2] http://marc.info/?l=openbsd-misc&m=148794264012274&w=2

adamsteen commented 7 years ago

Just a note to help others

While the best solution is worked out, OpenBSD users can continue to use stack, even without using only the system install ghc.

if you build a binary-dist and then specify it in your config.yaml, you can then use stack setup to installed it.

This is how i do it. On a fresh install of OpenBSD Current. Prerequisite:

/home is marked as wxallowed in /etc/fstab
the datasize-cur for staff in etc/login.conf is set to 4096M (it could be lower but i haven't bothered to find out)
and /etc/doas.conf has a line similar to "permit nopass setenv { PKG_CACHE PKG_PATH } adamsteen cmd pkg_add" (where adamsteen is your username)

Then I run the following script

#!/bin/sh
rm -rf $HOME/tmp && \
rm -rf $HOME/bin && \
rm -rf $HOME/ghc && \
mkdir tmp && \
mkdir bin && \
export PKG_PATH=http://mirror.internode.on.net/pub/OpenBSD/snapshots/packages/amd64/ && \
doas pkg_add ghc gmake 'autoconf-2.69p2' 'automake-1.15p0' gmp alex happy git xz gtar-- && \
ln -sf /usr/local/bin/python2.7 $HOME/bin/python && \
ln -sf /usr/local/bin/python2.7-2to3 $HOME/bin/2to3 && \
ln -sf /usr/local/bin/python2.7-config $HOME/bin/python-config && \
ln -sf /usr/local/bin/pydoc2.7 $HOME/bin/pydoc && \
export AUTOCONF_VERSION=2.69 && \
export AUTOMAKE_VERSION=1.15 && \
export TMP=$HOME/tmp && \
export TEMP=$HOME/tmp && \
git clone --recursive git://git.haskell.org/ghc.git && \
cd ghc && \
git checkout ghc-8.0 && \
git submodule update --init && \
./boot && \
./configure --with-iconv-libraries=/usr/local/lib --with-iconv-includes=/usr/local/include --with-gmp-libraries=/usr/local/lib --with-gmp-includes=/usr/local/include --with-ffi-libraries=/usr/local/lib --with-ffi-includes=/usr/local/include --with-system-libffi && \
gmake -j4 && \
gmake binary-dist && \
cd .. && \
mv ghc/ghc-* .

copy the resulting bindist to your development machine and include the following in your .stack/config.yaml file

setup-info:
  ghc:
    openbsd64:
      8.0.2.20170314:
        url: '/home/adamsteen/devl/ghc-bindist/ghc-8.0.2.20170314-x86_64-unknown-openbsd.tar.xz'

Note: with W^X now mandatory in OpenBSD back in may, I know its not the safest but i run my /home, /usr/local and /tmp marked as wxallowed.

I hope this helps other users out there!

seanwestfall commented 7 years ago

Thanks @adamsteen, A little more on OpenBSD's new wxallowed attribute. You can either set it by editing the /etc/fstab like so: 01020304050607.h /usr/local ffs rw,nodev,wxallowed 1 2 and rebooting, or you can temporary set a mount as wxallowed by running this command: mount -uo wxallowed /usr/local. (where /usr/local is the partition you'd like to affected, run mount to see them all)

In either case, I'm still having trouble getting ghc-8.0 to compile on OpenBSD 6.0. Here's the config and error I keep receiving after trying gmake:

Configure completed successfully.

   Building GHC version  : 8.0.2.20170314
          Git commit id  : d2d13a4f6750e30389552974bd7465712d522735

   Build platform        : x86_64-unknown-openbsd
   Host platform         : x86_64-unknown-openbsd
   Target platform       : x86_64-unknown-openbsd

   Bootstrapping using   : /usr/local/bin/ghc
      which is version   : 7.10.3

   Using gcc                 : /usr/bin/gcc
      which is version       : 4.2.1
   Building a cross compiler : NO
   hs-cpp       : /usr/bin/gcc
   hs-cpp-flags : -E -undef -traditional
   ld           : /usr/bin/ld
   nm           : /usr/bin/nm
   objdump      : /usr/bin/objdump
   Happy        : /home/goldfish/.cabal/bin/happy (1.19.5)
   Alex         : /home/goldfish/.cabal/bin/alex (3.2.1)
   Perl         : /usr/bin/perl
   sphinx-build :
   xelatex      :

   Using LLVM tools
      llc   :
      opt   :

   HsColour was not found; documentation will not contain source links

   Tools to build Sphinx HTML documentation available: NO
   Tools to build Sphinx PDF documentation available: NO
----------------------------------------------------------------------

For a standard build of GHC (fully optimised with profiling), type (g)make.

To make changes to the default build configuration, copy the file
mk/build.mk.sample to mk/build.mk, and edit the settings in there.

For more information on how to configure your GHC build, see
   http://ghc.haskell.org/trac/ghc/wiki/Building

$ gmake
===--- building phase 0
gmake --no-print-directory -f ghc.mk phase=0 phase_0_builds
gmake[1]: Nothing to be done for 'phase_0_builds'.
===--- building phase 1
gmake --no-print-directory -f ghc.mk phase=1 phase_1_builds
gmake[1]: Nothing to be done for 'phase_1_builds'.
===--- building final phase
gmake --no-print-directory -f ghc.mk phase=final all
Panic! libraries/Cabal/Cabal/dist-install/build/Distribution/Simple/Setup.hi exists, but libraries/Cabal/Cabal/dist-install/build/Distribution/Simple/Setup.dyn_o does not.
gmake[1]: *** [libraries/Cabal/Cabal/ghc.mk:4: libraries/Cabal/Cabal/dist-install/build/Distribution/Simple/Setup.dyn_o] Error 1
gmake: *** [Makefile:134: all] Error 2

also, I've been trying to build the latest master, but each time I get some different error, but maybe I should expect that from the latest. Here's the config and error message I receive when trying to build the latest master as of April 9, 2017:

Configure completed successfully.

   Building GHC version  : 8.3.20170409
          Git commit id  : 87377f74eec1567af741737b4b9034d06e3f0698

   Build platform        : x86_64-unknown-openbsd
   Host platform         : x86_64-unknown-openbsd
   Target platform       : x86_64-unknown-openbsd

   Bootstrapping using   : /usr/local/bin/ghc
      which is version   : 7.10.3

   Using (for bootstrapping) : /usr/bin/gcc
   Using gcc                 : gcc
      which is version       : 4.2.1
   Building a cross compiler : NO
   Unregisterised            : NO
   hs-cpp       : gcc
   hs-cpp-flags : -E -undef -traditional
   ld           : /usr/bin/ld
   nm           : /usr/bin/nm
   objdump      : /usr/bin/objdump
   Happy        : /home/goldfish/.cabal/bin/happy (1.19.5)
   Alex         : /home/goldfish/.cabal/bin/alex (3.2.1)
   Perl         : /usr/bin/perl
   sphinx-build :
   xelatex      :

   Using LLVM tools
      llc   :
      opt   :

   HsColour was not found; documentation will not contain source links

   Tools to build Sphinx HTML documentation available: NO
   Tools to build Sphinx PDF documentation available: NO
----------------------------------------------------------------------

For a standard build of GHC (fully optimised with profiling), type (g)make.

To make changes to the default build configuration, copy the file
mk/build.mk.sample to mk/build.mk, and edit the settings in there.

For more information on how to configure your GHC build, see
   http://ghc.haskell.org/trac/ghc/wiki/Building

$ gmake
===--- building phase 0
gmake --no-print-directory -f ghc.mk phase=0 phase_0_builds
gmake[1]: Nothing to be done for 'phase_0_builds'.
===--- building phase 1
gmake --no-print-directory -f ghc.mk phase=1 phase_1_builds
gcc -E -undef -traditional -P -DTOP='"/home/goldfish/yesod_test/ghc_compile_test/ghc"' -DFFI_INCLUDE_DIR=/usr/local/include -DFFI_LIB_DIR=/usr/local/lib '-DFFI_LIB=' -x c -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header rts/package.conf.in -o rts/dist/package.conf.inplace.raw
grep -v '^#pragma GCC' rts/dist/package.conf.inplace.raw | sed -e 's/""//g' -e 's/:[    ]*,/: /g' > rts/dist/package.conf.inplace
"inplace/bin/ghc-pkg" update --force rts/dist/package.conf.inplace
ghc-pkg: Couldn't open database /home/goldfish/yesod_test/ghc_compile_test/ghc/inplace/lib/package.conf.d for modification: /home/goldfish/yesod_test/ghc_compile_test/ghc/inplace/lib/package.conf.d/package.cache: GHC.PackageDb.readPackageDb: inappropriate type (Not a valid Unicode code point!)
gmake[1]: *** [rts/ghc.mk:540: rts/dist/package.conf.inplace] Error 1
gmake[1]: *** Deleting file 'rts/dist/package.conf.inplace'
gmake: *** [Makefile:125: all] Error 2
$

If anyone has seen these errors before and can point me in the right direction that would be very much appreciated.

adamsteen commented 7 years ago

@seanwestfall

Sorry i am only learning about the innards of GHC myself, i haven't see those errors. (Also i run Current)

if might be better posting to ghc-devs mailing list, they have been helpful to me in the past, I don't think this is the best spot.

also see Preparing and Building OpenBSD

Lastly, did you install Happy and Alex from ports or via cabal, i have always installed them via ports.

Blaisorblade commented 7 years ago

First, building GHC can be hard in general, also on Linux, even if you know the Makefile language, and even if you read all docs on the GHC wiki. I don't want to discourage you, but please read those docs. There are enough bugs to workaround.

This sort of error

Panic! libraries/Cabal/Cabal/dist-install/build/Distribution/Simple/Setup.hi exists, but libraries/Cabal/Cabal/dist-install/build/Distribution/Simple/Setup.dyn_o does not.

is often produced by pressing Ctrl-C after starting a build, so that only some of the output from one input are saved (here, the .hi, but not .dyn_o). This is a GHC bug and also happens during builds with stack or cabal. It happens to be often enough. As a workaround, remove the output which is there.

If you get that on a clean build (seems unlikely here): maybe the dyn_o file is just not being created. That's output for dynamic linking, but maybe it doesn't work on OpenBSD. Maybe the build.mk is not set correctly.

The second error is weirder. If you get it on a clean build, check permissions of the file where the build is failing. -- Paolo Giarrusso From smartphone, sorry for typos or excessive brevity

Il 10 apr 2017 03:30, "Sean Westfall" notifications@github.com ha scritto:

Thanks @adamsteen https://github.com/adamsteen, A little more on OpenBSD's new wxallowed attribute https://www.openbsd.org/faq/upgrade60.html. You can either set it by editing the /etc/fstab like so: 01020304050607.h /usr/local ffs rw,nodev,wxallowed 1 2 and rebooting, or you can temporary set a mount as wxallowed by running this command: mount -uo wxallowed /usr/local. (where /usr/local is the partition you'd like to affected, run mount to see them all)

In either case, I'm still having trouble getting ghc-8.0 to compile on OpenBSD 6.0. Here's the config and error I keep receiving after trying gmake :

Configure completed successfully.

Building GHC version : 8.0.2.20170314 Git commit id : d2d13a4f6750e30389552974bd7465712d522735

Build platform : x86_64-unknown-openbsd Host platform : x86_64-unknown-openbsd Target platform : x86_64-unknown-openbsd

Bootstrapping using : /usr/local/bin/ghc which is version : 7.10.3

Using gcc : /usr/bin/gcc which is version : 4.2.1 Building a cross compiler : NO hs-cpp : /usr/bin/gcc hs-cpp-flags : -E -undef -traditional ld : /usr/bin/ld nm : /usr/bin/nm objdump : /usr/bin/objdump Happy : /home/goldfish/.cabal/bin/happy (1.19.5) Alex : /home/goldfish/.cabal/bin/alex (3.2.1) Perl : /usr/bin/perl sphinx-build : xelatex :

Using LLVM tools llc : opt :

HsColour was not found; documentation will not contain source links

Tools to build Sphinx HTML documentation available: NO Tools to build Sphinx PDF documentation available: NO

For a standard build of GHC (fully optimised with profiling), type (g)make.

To make changes to the default build configuration, copy the file mk/build.mk.sample to mk/build.mk, and edit the settings in there.

For more information on how to configure your GHC build, see http://ghc.haskell.org/trac/ghc/wiki/Building

$ gmake ===--- building phase 0 gmake --no-print-directory -f ghc.mk phase=0 phase_0_builds gmake[1]: Nothing to be done for 'phase_0_builds'. ===--- building phase 1 gmake --no-print-directory -f ghc.mk phase=1 phase_1_builds gmake[1]: Nothing to be done for 'phase_1_builds'. ===--- building final phase gmake --no-print-directory -f ghc.mk phase=final all Panic! libraries/Cabal/Cabal/dist-install/build/Distribution/Simple/Setup.hi exists, but libraries/Cabal/Cabal/dist-install/build/Distribution/Simple/Setup.dyn_o does not. gmake[1]: [libraries/Cabal/Cabal/ghc.mk:4: libraries/Cabal/Cabal/dist-install/build/Distribution/Simple/Setup.dyn_o] Error 1 gmake: [Makefile:134: all] Error 2

Configure completed successfully.

Building GHC version : 8.3.20170409 Git commit id : 87377f74eec1567af741737b4b9034d06e3f0698

Build platform : x86_64-unknown-openbsd Host platform : x86_64-unknown-openbsd Target platform : x86_64-unknown-openbsd

Bootstrapping using : /usr/local/bin/ghc which is version : 7.10.3

Using (for bootstrapping) : /usr/bin/gcc Using gcc : gcc which is version : 4.2.1 Building a cross compiler : NO Unregisterised : NO hs-cpp : gcc hs-cpp-flags : -E -undef -traditional ld : /usr/bin/ld nm : /usr/bin/nm objdump : /usr/bin/objdump Happy : /home/goldfish/.cabal/bin/happy (1.19.5) Alex : /home/goldfish/.cabal/bin/alex (3.2.1) Perl : /usr/bin/perl sphinx-build : xelatex :

Using LLVM tools llc : opt :

HsColour was not found; documentation will not contain source links

Tools to build Sphinx HTML documentation available: NO Tools to build Sphinx PDF documentation available: NO

For a standard build of GHC (fully optimised with profiling), type (g)make.

To make changes to the default build configuration, copy the file mk/build.mk.sample to mk/build.mk, and edit the settings in there.

For more information on how to configure your GHC build, see http://ghc.haskell.org/trac/ghc/wiki/Building

$ gmake ===--- building phase 0 gmake --no-print-directory -f ghc.mk phase=0 phase_0_builds gmake[1]: Nothing to be done for 'phase_0_builds'. ===--- building phase 1 gmake --no-print-directory -f ghc.mk phase=1 phase_1_builds gcc -E -undef -traditional -P -DTOP='"/home/goldfish/yesod_test/ghc_compile_test/ghc"' -DFFI_INCLUDE_DIR=/usr/local/include -DFFI_LIB_DIR=/usr/local/lib '-DFFI_LIB=' -x c -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header rts/package.conf.in -o rts/dist/package.conf.inplace.raw grep -v '^#pragma GCC' rts/dist/package.conf.inplace.raw | sed -e 's/""//g' -e 's/:[ ]*,/: /g' > rts/dist/package.conf.inplace "inplace/bin/ghc-pkg" update --force rts/dist/package.conf.inplace ghc-pkg: Couldn't open database /home/goldfish/yesod_test/ghc_compile_test/ghc/inplace/lib/package.conf.d for modification: /home/goldfish/yesod_test/ghc_compile_test/ghc/inplace/lib/package.conf.d/package.cache: GHC.PackageDb.readPackageDb: inappropriate type (Not a valid Unicode code point!) gmake[1]: [rts/ghc.mk:540: rts/dist/package.conf.inplace] Error 1 gmake[1]: Deleting file 'rts/dist/package.conf.inplace' gmake: *** [Makefile:125: all] Error 2 $

If anyone ha seen these errors before and can point me in the right direction that would be very much appreciated.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/commercialhaskell/stack/issues/416#issuecomment-292829292, or mute the thread https://github.com/notifications/unsubscribe-auth/AARsqHFqHruvGho3kc8xvCRjI4ovgqWVks5ruYZSgaJpZM4FL9qe .

seanwestfall commented 7 years ago

Success guys! :tada: I let it compile over night, with only one core, just to be on the safe side, and it completed.

Though, I did the last step

$ cat /home/goldfish/.stack/config.yaml
templates:
  params: null
system-ghc: true
setup-info:
  ghc:
    openbsd64:
      8.0.2.20170314:
        url: '/home/goldfish/ghc-8.0.2.20170314-x86_64-unknown-openbsd.tar.xz'

and I'm not sure what this does actually. When I use stack setup I get this error:

$ stack setup
Preparing to install GHC to an isolated location.
This will not interfere with any system-level installation.
Did not find executable at specified path: /home/goldfish/.stack/programs/x86_64-openbsd/ghc-8.0.2.20170314.temp/ghc-8.0.2.20170314/configure

Error: Error encountered while configuring GHC with
         /home/goldfish/.stack/programs/x86_64-openbsd/ghc-8.0.2.20170314.temp/ghc-8.0.2.20170314/configure --prefix=/home/goldfish/.stack/programs/x86_64-openbsd/ghc-8.0.2.20170314/
         run in /home/goldfish/.stack/programs/x86_64-openbsd/ghc-8.0.2.20170314.temp/ghc-8.0.2.20170314/

The following directories may now contain files, but won't be used by stack:
  - /home/goldfish/.stack/programs/x86_64-openbsd/ghc-8.0.2.20170314.temp/
  - /home/goldfish/.stack/programs/x86_64-openbsd/ghc-8.0.2.20170314/

if the compiled completed correctly, shouldn't this return 8.0.2, and not 7.10.3?

$ pwd
/home/goldfish/ghc
$ ls
ANNOUNCE          bin               config.sub        install-sh        settings
HACKING.md        bindist-list      configure         iserv             settings.in
INSTALL.md        bindistprep       configure.ac      libffi            testsuite
LICENSE           bindisttest       distrib           libffi-tarballs   tmp
MAKEHELP.md       boot              docs              libraries         utils
Makefile          bootstrapping     driver            mk                validate
README.md         compiler          ghc               nofib
Vagrantfile       config.guess      ghc.mk            packages
aclocal.m4        config.log        includes          rts
autom4te.cache    config.status     inplace           rules
$ ghc --vresion
ghc: unrecognised flag: --vresion

Usage: For basic information, try the `--help' option.
$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 7.10.3
$

Blaisorblade commented 7 years ago

if the compiled completed correctly, shouldn't this return 8.0.2, and not 7.10.3?

Not really. Only stack exec -- ghc --version will use the GHC that stack installs — stack won't install it globally and mess with your system.

Anyway, stack setup says it failed and it seems a correct error.

To fix that failure:

You need to make sure you have xz installed (not default in OpenBSD). This seems the case.
I recommend removing /home/goldfish/.stack/programs/x86_64-openbsd/ghc-8.0.2.2017031* (both folders), since the install failed.
how did you prepare /home/goldfish/ghc-8.0.2.20170314-x86_64-unknown-openbsd.tar.xz? It needs to be a bindist like the ones from GHC, with a configure file for setting the prefix later. According to https://ghc.haskell.org/trac/ghc/wiki/Building/Installing#Buildingandinstallingfromabinarydistribution, you need gmake binary-dist to create that file.
If a proper binary dist fails install, try again with stack setup -v to get more verbose logs—take a look for clues and/or post them here.
If instead you get a binary distro that works, and is for an OpenBSD version not otherwise supported, feel free to share! (I think Stack would benefit from builds for stable GHC versions, if possible, but hopefully that should be easier to achieve now).

ketzacoatl commented 7 years ago

How feasible is it to support multiple GHC versions in ports?

mrijkeboer commented 7 years ago

I asked this question a few month ago on the ports mailing list. I received the following answer from the OpenBSD GHC maintainer (kili): http://marc.info/?l=openbsd-ports&m=147224433914597&w=2

seanwestfall commented 7 years ago

@Blaisorblade Already mentioned this to @adamsteen But the only step I needed to do to finish up was run gmake install as root to finish up the install process and install and move the compiled GHC to the global executable. What would be left to upload GHC 8.0.2 to the OpenBSD 6.0 packages/ports list, would this be up to kili? Compiling GHC 8.0.2 on OpenBSD 6.0 hasn't been to hard.

Though, I guess next steps is now compiling GHC 8.2.0 on OpenBSD 6.1.

Blaisorblade commented 7 years ago

@seanwestfall No: gmake install as root does a global install, and it does not fix stack setup. Stack setup can install multiple GHC versions, but that's only possible because it has its own installation method, and installs things in a non-global location. By default, stack won't even use the globally installed GHC (though it can be configured for that). That's why stack would need a binary dist, and why I provided instructions to build one.

I'm sure uploasing GHC to ports would help GHC users on BSD, but it's not enough for stack official support.

ketzacoatl commented 7 years ago

Right, the real problem here is that we don't (yet) have an easy way to build / host multiple versions of GHC for each OpenBSD release. Any proposals for realistic methods to do so?

Blaisorblade commented 7 years ago

Right, the real problem here is that we don't (yet) have an easy way to build / host multiple versions of GHC for each OpenBSD release. Any proposals for realistic methods to do so?

@ketzacoatl I see supporting multiple releases in ports is an organizational problem. But support in port is not too relevant to stack.

Both Stack (and I think GHC headquarters) could host community-supported working bindists. The bigger issue, at this point, is autodetecting the OpenBSD bindist to use: https://github.com/commercialhaskell/stack/issues/2822#issuecomment-300591681

Stack setup already supports some (https://github.com/fpco/stackage-content/blame/master/stack/stack-setup-2.yaml)—apparently, the ones from Karel Gardas, hosted by GHC headquarters (https://www.haskell.org/ghc/download_ghc_8_0_1#openbsd_x86_64, https://www.haskell.org/ghc/download_ghc_8_0_2#openbsd_x86_64).

adamsteen commented 7 years ago

@seanwestfall I have to agree with Blaisorblade here, i don't install GHC i built, but use stack to do its magic with my version of the bindist! I am happy with this until we can come up with a way for stack to support in natively. I use the same process with 8.2.1.

seanwestfall commented 7 years ago

@Blaisorblade Okay, yes you're right. For some reason, when I followed the instructions for adding the bindist file to the .stack/config.yaml it just produced the errors I reported earlier. I'll fiddle more with stack's config to figure out how to get it to work with the bindist, but running gmake install was the only way for me to know that the compile process could even produce a working GHC.

Blaisorblade commented 7 years ago

For anybody still interested: we might have all the pieces needed. But I got no time to coordinate testing and integration.

We're making progress on #2822 to have Stack select a different GHC bindist on OpenBSD for each major release. The needed Haskell coding might be done, but is untested (https://github.com/commercialhaskell/stack/compare/master...check-openbsd-release). I also sketched the needed changes to Stack config to find the bindists (https://github.com/fpco/stackage-content/pull/25) — you can try them out locally via https://docs.haskellstack.org/en/stable/yaml_configuration/#setup-info.

As usual, I can't actually build bindists. But hopefully those changes contain all the Stack expertise needed.

seanwestfall commented 7 years ago

Good work @Blaisorblade. Though what about what's listed on the ports pages? e.g. https://mirrors.sonic.net/pub/OpenBSD/6.1/packages/amd64/ghc-7.10.3p7.tgz, https://mirrors.sonic.net/pub/OpenBSD/6.0/packages/amd64/ghc-7.10.3p4.tgz Should we care about people getting GHC that way?

Blaisorblade commented 7 years ago

Should we care about people getting GHC that way?

Stack supports usual system-installed GHCs, but it can't manage those. IIUC, those users are already supported—they just need to setup https://docs.haskellstack.org/en/stable/yaml_configuration/#system-ghc to true, like on all other platforms.

Though what about what's listed on the ports pages?

In case you wonder: those archives can't be used as bindists, because their contents are different (for instance, no configure and Makefile for installation). It might just be a matter of packaging; I still guess building a bindist from scratch is much easier than trying to adjust those.

ketzacoatl commented 7 years ago

Interestingly, the 8.2.1 release notes mentions the following

GHC HQ now builds FreeBSD and OpenBSD distributions for amd64; this comes after many years of these distributions being faithfully provided by Karel Gardas and Pali Gabor Janos, who we should heartily thank for their contributions. GHC HQ building these distributions ourselves will allow us to more quickly ship distributions to users by eliminating the need for a long lag time between source release availability and having all binary distributions available.

https://ghc.haskell.org/trac/ghc/blog/ghc-8.2.11-released

Blaisorblade commented 7 years ago

OK, I've now marked #3184 as fixing this since that's the major stumbling block. I'm not sure that's the only open problem, but this issue is too big—I've forked #3313 out of this to highlight possibly-important remaining steps.