Register multiple GHC versions per workspace

[brendanhay]

I'd like to register multiple GHC versions per WORKSPACE and then use --config, --host_platform, or something similar in spirit to build using the desired compiler version. Given that the stack_snapshot repository rule has no way to conditionally set/select the snapshot attribute to ensure we get the right version of base et al. for the compiler, is what follows a sane (modulo some rules/abstractions) way of achieving this:

# WORKSPACE
haskell_register_ghc_nixpkgs(
    name = "ghc884",
    version = "8.8.4",
    attribute_path = "haskell.compiler.ghc884",
    repository = "@nixpkgs",
    exec_constraints = ["@//tools/constraints:ghc884"],
    target_constraints = ["@//tools/constraints:ghc884"],
)

haskell_register_ghc_nixpkgs(
    name = "ghc8107",
    version = "8.10.7",
    attribute_path = "haskell.compiler.ghc8107",
    repository = "@nixpkgs",
    exec_constraints = ["@//tools/constraints:ghc8107"],
    target_constraints = ["@//tools/constraints:ghc8107"],
)

STACK_PACKAGES = [ ... ]

stack_snapshot(
    name = "stackage_ghc884",
    snapshot = "lts-16.31", 
    stack_snapshot_json = "//:stackage-lts-16.31.json",
    packages = STACK_PACKAGES,
)

stack_snapshot(
    name = "stackage_ghc8107",
    snapshot = "lts-18.10", 
    stack_snapshot_json = "//:stackage-lts-18.10.json",
    packages = STACK_PACKAGES,
)

# haskell/foo/BUILD.bzl

PACKAGES = [ "base", "filepath", "text" ]

haskell_library(
    name = "foo",
    srcs = ["Foo.hs"],
    deps = select({
        "@//tools/constraints:ghc884": ["@stackage_ghc884//:%s" % pkg for pkg in PACKAGES],
        "@//tools/constraints:ghc8107": ["@stackage_ghc8107//:%s" % pkg for pkg in PACKAGES],
    })
)

# tools/constraints/BUILD.bzl

constraint_setting(name = "ghc_version")

constraint_value(
    name = "ghc884",
    constraint_setting = ":ghc_version",
    visibility = ["//visibility:public"],
)

constraint_value(
    name = "ghc8107",
    constraint_setting = ":ghc_version",
    visibility = ["//visibility:public"],
)

# tools/platforms/BUILD.bzl

platform(
    name = "ghc884",
    constraint_values = [
        "//tools/constraints:ghc884",
    ],
    parents = ["@io_tweag_rules_nixpkgs//nixpkgs/platforms:host"],
    visibility = ["//visibility:public"],
)

platform(
    name = "ghc8107",
    constraint_values = [
        "//tools/constraints:ghc8107",
    ],
    parents = ["@io_tweag_rules_nixpkgs//nixpkgs/platforms:host"],
    visibility = ["//visibility:public"],
)

Are there perhaps living breathing examples of this in the wild?

(One particular annoyance with the above code is poor interaction with tooling like gazelle_cabal, but that's neither here nor there.)

[aherrmann]

That's a good question. Thanks for the code examples!

It's not exactly the same use-case, but we have an example for cross-compilation here that involves two GHC toolchains, one for x86_64 and one for ARM. That may be helpful, if you haven't seen it, yet.

---

Given that the stack_snapshot repository rule has no way to conditionally set/select the snapshot attribute to ensure we get the right version of base et al. for the compiler

Something worth noting here is that core-packages are not fetched from Stackage, but taken from the packages shipped with GHC. So, you'll automatically use the version of base that comes with GHC independent of what the snapshot says. Of course, you may still need to have two stack_snapshots to be sure that other packages match GHC and its core-package versions.

Yes, you can select between the two stack_snapshots at the use-site as you show. You could also add a layer of indirection by defining alias targets for all Stackage packages and placing the selects on these aliases. This way you don't need to repeat the select over and over throughout your code-base. Instead, your regular targets just depend on these aliases. If you place these aliases into an external repository, then you can use gazelle_cabal's gazelle:cabal_haskell_package_repo directive to point it to this "alias" repository.

---

It is possible to use platform constraints to switch between compilers as you show. However, it's perhaps a bit of a misuse of the feature, as these are not really platform differences. It also brings the problem of combinatorial explosion if you have other compilers or tools who's version you would like to switch on in this way.

An alternative is to switch on a build setting. I've created a toy example of this idea a while ago here. The trick is to add a config setting for the compiler version and then select on the toolchain rule's toolchain attribute as shown here. Note, you don't necessarily need to patch rules_haskell for this. Instead, I think, you could replicate the toolchain rules generated by haskell_register_ghc_nixpkgs in your project and make sure to register them first (Bazel's toolchain selection takes precedence into account).

[brendanhay]

Awesome, thanks a lot for the pointers!

[brendanhay]

For posterity if anyone else stumbles upon this, here's a condensed example after cribbing @aherrmann's work:

# WORKSPACE

...

load("@rules_haskell//haskell:nixpkgs.bzl", "haskell_register_ghc_nixpkgs")

# Register custom toolchain prior to rules_haskell toolchains.
register_toolchains("//tools/ghc:toolchain")

haskell_register_ghc_nixpkgs(
    name = "ghc865",
    attribute_path = "haskell.compiler.ghc865",
    repository = "@nixpkgs",
    version = "8.6.5",
)

haskell_register_ghc_nixpkgs(
    name = "ghc884",
    attribute_path = "haskell.compiler.ghc884",
    repository = "@nixpkgs",
    version = "8.8.4",
)

haskell_register_ghc_nixpkgs(
    name = "ghc8107",
    attribute_path = "haskell.compiler.ghc8107",
    repository = "@nixpkgs",
    version = "8.10.7",
)

# tools/ghc/BUILD.bazel

load("@bazel_skylib//rules:common_settings.bzl", "string_flag")

string_flag(
    name = "version",
    build_setting_default = "8107",
    values = [
        "865",
        "884",
        "8107",
    ],
)

config_setting(
    name = "865",
    flag_values = {
        ":version": "865",
    },
)

config_setting(
    name = "884",
    flag_values = {
        ":version": "884",
    },
)

config_setting(
    name = "8107",
    flag_values = {
        ":version": "8107",
    },
)

# `haskell_register_ghc_nixpkgs` declares repositories with names formatted as:
# `@{name}_ghc_nixpkgs_haskell_toolchain`.
toolchain(
    name = "toolchain",
    toolchain_type = "@rules_haskell//haskell:toolchain",
    toolchain = select({
    "//tools/ghc:865":  "@ghc865_ghc_nixpkgs_haskell_toolchain//:toolchain-impl",
    "//tools/ghc:884":  "@ghc884_ghc_nixpkgs_haskell_toolchain//:toolchain-impl",
    "//tools/ghc:8107": "@ghc8107_ghc_nixpkgs_haskell_toolchain//:toolchain-impl",
    }),
)

# Usage
bazel build --//tools/ghc:version=865  //...
bazel build --//tools/ghc:version=884  //...
bazel build --//tools/ghc:version=8107 //...

[brendanhay]

And an attempt at implementing your stack_snapshot suggestion using aliases in an external repository:

# tools/haskell.bzl

load("@rules_haskell//haskell:cabal.bzl", "stack_snapshot")

def _snapshot_repo_name(snapshot):
    return "stackage-{}".format(snapshot)

def _stack_snapshot_alias_impl(repository_ctx):
    content = ['package(default_visibility = ["//visibility:public"])']

    for pkg in repository_ctx.attr.packages:
        conditions = []

        for ghc, snapshot in repository_ctx.attr.snapshots.items():
            conditions.append('"@{ghc}": "@{snapshot}//:{pkg}"'.format(
                pkg = pkg,
                ghc = ghc,
                snapshot = _snapshot_repo_name(snapshot),
            ))

        content.append("""
alias(
    name = "{pkg}",
    actual = select({{
        {conditions}
    }})
)""".format(
            pkg = pkg,
            conditions = ",\n        ".join(conditions),
        ))

    repository_ctx.file(
        "BUILD.bazel",
        executable = False,
        content = "\n".join(content),
    )

stack_snapshot_alias = repository_rule(
    implementation = _stack_snapshot_alias_impl,
    attrs = {
        "snapshots": attr.label_keyed_string_dict(),
        "packages": attr.string_list(),
    },
)

def stack_snapshots(name, snapshots, packages, **kwargs):
    for ghc, snapshot in snapshots.items():
        repo_name = _snapshot_repo_name(snapshot)

        stack_snapshot(
            name = repo_name,
            snapshot = snapshot,
            packages = packages,
            stack_snapshot_json = "//:{}-snapshot.json".format(repo_name),
            **kwargs
        )

    stack_snapshot_alias(
        name = name,
        snapshots = snapshots,
        packages = packages,
    )

# WORKSPACE

load("//tools:haskell.bzl", "stack_snapshots")

stack_snapshots(
    name = "stackage",
    snapshots = {
        "//tools/ghc:865": "lts-16.31",
        "//tools/ghc:884": "lts-18.10",
        "//tools/ghc:8107": "lts-18.10",
    },
    packages = [
        "QuickCheck",
        "aeson",
        "attoparsec",
        "base",
        "bifunctors",
        "bytestring",
        ...
    ],
    extra_deps = {
        "zlib": ["@zlib.dev//:zlib"],
        "digest": ["@zlib.dev//:zlib"],
    },
    setup_deps = {
        "xml-conduit": ["@stackage//:cabal-doctest"],
    },
    tools = [
        "@nixpkgs_alex//:bin/alex",
        "@nixpkgs_happy//:bin/happy",
    ],
)

> bazel build --//tools/ghc:version=884 @stackage//:bytestring
INFO Analyzed target @stackage//:bytestring
...

[aherrmann]

@brendanhay Thank you for sharing your setup, that looks great! Would you be interested in contributing a section to the use-cases docs for this?

[brendanhay]

Sure, I'll take a wild stab at it shortly.