Does not support build-tools like alex and happy if not installed external to cabal-dev

[joshtriplett]

I tried to use cabal-dev to build a package which depends on alex and happy. At first, I had alex and happy in the build-tools field of the project's cabal file, but not in build-depends. This caused the build of that project to simply fail to find alex and happy. I then added alex and happy to build-depends. cabal-dev then built and installed alex and happy into the cabal-dev directory (they appear in cabal-dev/bin), but then when it went to build the package needing them it said:

cabal: At least the following dependencies are missing: alex ==3.0.1, happy ==1.18.6

Those versions precisely match the versions cabal-dev had just installed.

[creswick]

The problem is that cabal, as invoked by cabal-dev, doesn't (usually) know to look in <sandbox>/bin for build tools, it just uses your path (which may contain <sandbox>/bin, in which case you probably wouldn't be having this problem).

There are a few different thoughts (at least!) on how to solve this problem:

1. Just add <sandbox>/bin to the path that the sub-cabal processes see.
2. Claim these tools are part of the build system dependencies, and as such, they shouldn't be installed into the same sandbox as the program dependencies, since there is no fundamental relationship between the those dependency sets, and punt on the problem. (effectively what we've done so far).
3. no. 2, but instead of punting, use a separate sandbox (but we don't have a good way to identify build dependencies -- although build-tools is a special case in which we can.)
4. no. 3, but create a general way to specify build system dependencies - probably requiring cabal / cabal-install patching.

The first option is the simplest, but I don't like it because there is too much opportunity for running things you didn't mean to run. Granted, if you're using cabal, you're running arbitrary code anyway, but adding paths that aren't in $PATH seems like the kind of change that eventually costs days and days of frustrated debugging when a build refuses to use the latest version of happy that came with the GHC you just installed. (exacerbated because the standard things, like which happy and happy --version report what you expected when you run them at a shell, but they would do something else when you run cabal-dev install)

I think the last option is the right one, and there is a GSOC ticket for it: (http://hackage.haskell.org/trac/summer-of-code/ticket/1602), but we might be able to do something in the interim.

Ideas?

[joshtriplett]

I actually tried putting the sandbox's bin directory on my path, but that didn't change the result at all.

[creswick]

Ah, right.

As you're probably aware, cabal doesn't track the installation of executables -- only libraries. Adding an exe to the build depends will force it to be installed (every time you cabal install), but it doesn't seem to satisfy the dependency.

I did some experimentation with BNFC, another code generator from hackage (because I didn't want to tweak with my aptly-named happy installs). Here's my Setup.hs, which is important if you want to reproduce this (since BNFC isn't one of cabal's provided Program's, so you have to define a Program value for it and hook it into the build.):

-- Don't forget:
-- Build-type: Custom
--
-- Build-dependencies: BNFC == 2.4.2.0
-- 
-- and:
-- Build-tools: bnfc == 2.4.2.0
--
import Distribution.Simple
import Distribution.Simple.Program.Types
import Distribution.Simple.Setup
import Distribution.Simple.UserHooks
import Distribution.Simple.Utils ( rawSystemExit, warn, debug
                                 , findProgramVersion, notice )
import Distribution.Verbosity ( Verbosity )

import System.Directory ( getCurrentDirectory, setCurrentDirectory,
                          removeFile )
import System.Cmd       ( rawSystem )

main = defaultMainWithHooks simpleUserHooks { hookedPrograms = [bnfcProgram] }

bnfcProgram :: Program
bnfcProgram = (simpleProgram "bnfc") {
  programFindVersion = findProgramVersion "--numeric-version" id
  }

This won't work, because cabal will never realize that the build-depends requirement is satisfied.

For reference (and to help with debugging), the 'dependency not installed' and 'build tool not available' errors are:

# Build-Depends is missing:
cabal: At least the following dependencies are missing:
BNFC ==2.4.2.0

# Build-tool is missing:
cabal: Unknown build tool bnfc

The first error is the one I kept getting, despite clearly seeing cabal build bnfc. Once the build tools are installed, then the $PATH hack seems to work, but only if you remove the build depends for the required tool(s). I suspect you're running into the first style error with happy -- cabal just doesn't realize that happy is installed, even though that's what it just did. I was able to get cabal to use the sandboxed version of bnfc by tweaking my PATH (export PATH=./cabal-dev/bin:$PATH) but only after the aforementioned installations and cabal file tweaking.

You can install the build tools manually to the sandbox to get around some of the ugliness, but the versions won't be updated (obviously). That may work for the moment, but it's not very satisfactory...