Haskell binary building with cabal new-build

blackgnezdo commented 5 years ago

While looking at Haskell binary ports, a couple of approaches seem possible.

1) Embed Haskell libraries into ports, build binaries from them. 2) Build binaries with cabal new-build

The status quo is 1. This has a few benefits: it exists, libraries are reused by multiple ports reducing build costs. There are drawbacks: the ports tree contains a duplicate (and sometimes inconsistent) specification of Hackage information. Because of this, upgrades are a big pain and there's a version of DLL (cabal) hell in that a single compatible library version needs to be chosen for all ports.

An alternative is 2. Many hs-* library ports could go away by making each binary port (e.g. darcs, shellcheck) build their own library dependencies in its sandbox. This will reduce maintenance costs and avoid any possibility of version conflicts. There are drawbacks: it doesn't exist, the same library will get built for each binary using it. We also won't be able to switch to a completely clean 2. Some ports (like xmonad) use Haskell build as a configuration mechanism. We still need to provide a subset of ports-embedded libraries in the style of 1.

If we are to keep doing 1, we should automate maintenance as much as possible. portgen is an option and so is building a port targeting version of cabal2bazel (once brought up to date)

If the majority of OpenBSD users only care about using the binaries, I believe we will serve them better by doing 2 for the binary providing ports.

Some unknowns apply to design 2:

how well we can specify the Hackage snapshot
how hermetic we can make it (force tarball fetching to happen before running cabal new-build)

Ports with Haskell binaries:

% for i in $(</tmp/all-hs ); do grep '^@bin' $i/pkg/PLIST*>/dev/null && echo $i; done
devel/alex
devel/cabal-install
devel/cpphs
devel/darcs
devel/gmc4cc
devel/happy
devel/hasktags
devel/shellcheck
games/hedgewars
net/hpodder
textproc/hs-HaXml
x11/xmonad
x11/xmobar

All ports (including sub-packages) dependent on ghc:

sqlite> select count(fullpkgpath) from ports where build_depends like '%ghc%';
123

blackgnezdo commented 4 years ago

Some progress in my understanding. cabal-install can generate plan.json which is a good way to construct the build graph which could later be turned into ports. E.g. given this cabal.project file which enumerates a subset of the packages above:

packages: https://hackage.haskell.org/package/xmobar-0.33/xmobar-0.33.tar.gz
          https://hackage.haskell.org/package/xmonad-0.15/xmonad-0.15.tar.gz
          https://hackage.haskell.org/package/darcs-2.14.4/darcs-2.14.4.tar.gz
          https://hackage.haskell.org/package/alex-3.2.5/alex-3.2.5.tar.gz
          https://hackage.haskell.org/package/happy-1.19.12/happy-1.19.12.tar.gz

allow-newer: setlocale:base xmobar:base

package xmobar
    flags: with_xft with_mpd with_mpris with_inotify
    extra-include-dirs: /usr/X11R6/include
    extra-lib-dirs: /usr/local/lib/inotify
    ld-options: -Wl,-rpath=/usr/local/lib/inotify

package hinotify
        extra-include-dirs: /usr/local/include/inotify
        extra-lib-dirs: /usr/local/lib/inotify

package X11
  extra-lib-dirs: /usr/X11R6/lib
  extra-include-dirs: /usr/X11R6/include

One can run this command:

cabal v2-build --dry --project-file ./cabal.project -w /usr/local/bin/ghc all

and out comes a nice dist-newstyle/cache/plan.json which could then go into cabal-plan package for turning into ports.

blackgnezdo commented 4 years ago

FWIW, FreeBSD ports have been using something cabal-based since a month before I created this issue. We should be able to check if they have any regrets and learn of sharp corners from their experience.

FreeBSD documentation on creating cabal based ports.

blackgnezdo commented 4 years ago

There's a choice to make when building the framework:

Single "slice" through cabal packages for all Haskell ports
Per-port cabal configuration

Single configuration

Minimizes the number of distfiles to mirror.
With sufficient effort could be made to reuse build products (provided compatible cabal flags)

Per-port configuration

Removes cross-port constraints (easier port creation, port upgrades)

Flexibility wins

I'm leaning toward using per-port configuration for ease of independent port maintenance.

blackgnezdo commented 4 years ago

Mostly implemented in https://github.com/blackgnezdo/ports/commit/6d26d47c60fbd1fe46c5b39ac638be2b53e81d7e

blackgnezdo / ports