Compile failure with `proto-lens-combinators-0.1.0.8`

[hvr]

In order, the following will be built (use -v for more details):
 - proto-lens-combinators-0.1.0.8 {proto-lens-combinators-0.1.0.8-inplace} (lib:proto-lens-combinators) (first run)
[1 of 1] Compiling Main             ( /tmp/matrix-worker/1501476906/dist-newstyle/build/x86_64-linux/ghc-8.2.1/proto-lens-combinators-0.1.0.8/setup/setup.hs, /tmp/matrix-worker/1501476906/dist-newstyle/build/x86_64-linux/ghc-8.2.1/proto-lens-combinators-0.1.0.8/setup/Main.o )
Linking /tmp/matrix-worker/1501476906/dist-newstyle/build/x86_64-linux/ghc-8.2.1/proto-lens-combinators-0.1.0.8/setup/setup ...
<<ghc: 195021320 bytes, 90 GCs, 9019646/22334576 avg/max bytes residency (7 samples), 57M in use, 0.001 INIT (0.001 elapsed), 0.123 MUT (1.892 elapsed), 0.185 GC (0.185 elapsed) :ghc>>
Configuring proto-lens-combinators-0.1.0.8...
<<ghc: 190663776 bytes, 110 GCs, 12726334/39654304 avg/max bytes residency (8 samples), 102M in use, 0.001 INIT (0.001 elapsed), 0.101 MUT (0.101 elapsed), 0.218 GC (0.221 elapsed) :ghc>>
==========
Error: couldn't find the executable "proto-lens-protoc" in your $PATH.
    Please file a bug at https://github.com/google/proto-lens/issues .
==========
Missing executable "proto-lens-protoc"
CallStack (from HasCallStack):
  error, called at src/Data/ProtoLens/Setup.hs:297:13 in proto-lens-protoc-0.2.2.0-6fbfcc9fefb6f837231240070e1fad9e51f23d5d830dd28e2a4fa31f1e705ca4:Data.ProtoLens.Setup

[judah]

Thank you for the report. Can you please give:

• The value of your $PATH
• The result of which proto-lens-protoc
• The command that you ran to produce that error
• If you used cabal, the output of cabal --version

I just verified that cabal install proto-lens-combinators worked on my machine. My $PATH includes ~/.cabal/bin.

[hvr]

$PATH does not contain proto-lens-protoc nor is proto-lens-protoc installed, as cabal new-build had no indication it needed to auto-install and provide proto-lens-protoc on $PATH (in order to that, you need to use build-tools-depends; build-depends or setup-depends only specifies to bring library components into scope). In the new-build paradigm, there is no auto-populated ~/.cabal/bin anymore, as you need to be able to satisfy multiple versions of executables simultaneously.

I'm currently investigating how we can get this to work in cabal new-build (which as you may know is going to become the default in cabal-3.0 - so we still have time to figure out how to make this work), as it appears that it's not enough to add

  build-tool-depends: proto-lens-protoc:proto-lens-protoc >= 0.1 && < 0.3

to the library section, as the build-tool is already required at Setup.hs time.

So I'm afraid that this can't be addressed for cabal-2.0 users yet, and we may need to extend the custom-setup grammar in cabal-2.2 to express this kind of dependency; I'll keep you posted.

---

PS: The commands to reproduce this more or less how https://matrix.hackage.haskell.org/package/proto-lens-combinators does it:

$ cabal --version
cabal-install version 2.1.0.0
compiled using version 2.1.0.0 of the Cabal library

$ cabal get proto-lens-combinators
Unpacking to proto-lens-combinators-0.1.0.8/

$ cd proto-lens-combinators-0.1.0.8/

$ cabal new-build
Resolving dependencies...
Build profile: -w ghc-8.2.1 -O1
In order, the following will be built (use -v for more details):
 - proto-lens-combinators-0.1.0.8 {proto-lens-combinators-0.1.0.8-inplace} (lib:proto-lens-combinators) (first run)
[1 of 1] Compiling Main             ( /tmp/proto-lens-combinators-0.1.0.8/dist-newstyle/build/x86_64-linux/ghc-8.2.1/proto-lens-combinators-0.1.0.8/setup/setup.hs, /tmp/proto-lens-combinators-0.1.0.8/dist-newstyle/build/x86_64-linux/ghc-8.2.1/proto-lens-combinators-0.1.0.8/setup/Main.o )
Linking /tmp/proto-lens-combinators-0.1.0.8/dist-newstyle/build/x86_64-linux/ghc-8.2.1/proto-lens-combinators-0.1.0.8/setup/setup ...
Configuring proto-lens-combinators-0.1.0.8...
==========
Error: couldn't find the executable "proto-lens-protoc" in your $PATH.
    Please file a bug at https://github.com/google/proto-lens/issues .
==========
Missing executable "proto-lens-protoc"
CallStack (from HasCallStack):
  error, called at src/Data/ProtoLens/Setup.hs:297:13 in proto-lens-protoc-0.2.2.0-6fbfcc9fefb6f837231240070e1fad9e51f23d5d830dd28e2a4fa31f1e705ca4:Data.ProtoLens.Setup

---

PS2: Here's what happens if I add build-tool-depends to the library stanza:

Build profile: -w ghc-8.2.1 -O1
In order, the following will be built (use -v for more details):
 - proto-lens-combinators-0.1.0.8 {proto-lens-combinators-0.1.0.8-inplace} (lib:proto-lens-combinators) (first run)
./proto-lens-combinators.cabal has been changed. Re-configuring with most
recently used options. If this fails, please run configure manually.
Configuring proto-lens-combinators-0.1.0.8...
Preprocessing library for proto-lens-combinators-0.1.0.8..
Building library for proto-lens-combinators-0.1.0.8..
Can't run protoc; run 'cabal configure' first.
CallStack (from HasCallStack):
  error, called at src/Data/ProtoLens/Setup.hs:155:24 in proto-lens-protoc-0.2.2.0-6fbfcc9fefb6f837231240070e1fad9e51f23d5d830dd28e2a4fa31f1e705ca4:Data.ProtoLens.Setup
  sDistHook, called at ./Distribution/Simple/UserHooks.hs:113:5 in Cabal-2.0.0.2-0da1efa03ad749960141ea36de1dddd541a1cbe6060c2da24f506e4b072b4bc9:Distribution.Simple.UserHooks
  sdistAction, called at ./Distribution/Simple.hs:182:50 in Cabal-2.0.0.2-0da1efa03ad749960141ea36de1dddd541a1cbe6060c2da24f506e4b072b4bc9:Distribution.Simple
  defaultMainHelper, called at ./Distribution/Simple.hs:128:42 in Cabal-2.0.0.2-0da1efa03ad749960141ea36de1dddd541a1cbe6060c2da24f506e4b072b4bc9:Distribution.Simple
  defaultMainWithHooks, called at src/Data/ProtoLens/Setup.hs:94:7 in proto-lens-protoc-0.2.2.0-6fbfcc9fefb6f837231240070e1fad9e51f23d5d830dd28e2a4fa31f1e705ca4:Data.ProtoLens.Setup

which points to the code-fragment

    , sDistHook = \p maybe_l h f -> case maybe_l of
            Nothing -> error "Can't run protoc; run 'cabal configure' first."
            Just l -> do
                        generate p l
                        sDistHook hooks (fudgePackageDesc l p) maybe_l h f

[judah]

Thanks for the detailed explanation. For your "PS2", I tracked this down to --list-sources:

$ cabal get proto-lens-combinators-0.1.0.8
$ cd proto-lens-combinators-0.1.0.8
$ cabal sdist --list-sources=output
Can't run protoc; run 'cabal configure' first.
CallStack (from HasCallStack):
error, called at src/Data/ProtoLens/Setup.hs:155:24 in
proto-lens-protoc-0.2.2.0-62akSW3yVjMghzUMuuS0D:Data.ProtoLens.Setup

The current implementation of sdist in Data.ProtoLens.Setup doesn't work unless cabal configure is run first. We'll need to look into it further to tell whether this is something that module can fix, or if we'd want the behavior of cabal-install itself to change.

[Ericson2314]

I see the error message I get is now in a head #if: https://github.com/google/proto-lens/blob/master/proto-lens-protoc/src/Data/ProtoLens/Setup.hs#L188-L206 . Does that mean this problem is solved?

[chshersh]

@Ericson2314 Your link is broken, unfortunately 😞

Also, I still see the following error:

Can't run protoc; run 'cabal configure' first.
CallStack (from HasCallStack):
  error, called at src/Data/ProtoLens/Setup.hs:191:24 in prt-lns-prtc-0.3.1.0-2003cf77:Data.ProtoLens.Setup
  sDistHook, called at libraries/Cabal/Cabal/Distribution/Simple/UserHooks.hs:113:5 in Cabal-2.2.0.1:Distribution.Simple.UserHooks

when I'm trying to build project with cabal new-build.

[mpickering]

I think that the 0.4 version fixes the problem of new-build not working at all. Will it be released on hackage soon?

[judah]

@mpickering are you sure it fixes the issue? I just tried cabal new-build on the latest master of proto-lens and got the same error as originally reported:

$ cabal new-build proto-lens-protobuf-types
Build profile: -w ghc-8.4.3 -O1
In order, the following will be built (use -v for more details):
 - proto-lens-protobuf-types-0.3.0.3 (lib:proto-lens-protobuf-types) (first run)
==========
Error: couldn't find the executable "proto-lens-protoc" in your $PATH.
    Please file a bug at https://github.com/google/proto-lens/issues .
==========
Missing executable "proto-lens-protoc"
CallStack (from HasCallStack):
  error, called at src/Data/ProtoLens/Setup.hs:394:13 in proto-lens-setup-0.4.0.0-inplace:Data.ProtoLens.Setup

I'm hoping to release 0.4 next week; the current progress is being tracked in https://github.com/google/proto-lens/wiki/Release-Plan .

[mpickering]

I made sure proto-lens-protoc was on my path then the error in https://github.com/google/proto-lens/issues/110#issuecomment-411006689 is fixed.

[judah]

Update: I've released proto-lens-protoc-0.3.1.1 which contains the fix in question. @mpickering can you please try it out and see whether it works for you?

Also, I verified manually that using build-tool-depends: proto-lens-protoc:proto-lens-protoc (as suggested in https://github.com/google/proto-lens/issues/110#issuecomment-318984283) should prevent you from needing to put proto-lens-protoc in your path.

I think this issue can be closed once:

1. We document that build-tool-depends is required for cabal new-build
2. We add it to packages like proto-lens-protobuf-types
3. We add another build job for cabal new-build in Travis CI

It's unfortunate that this complicates proto-lens's Cabal UX. Long-term something like haskell/cabal#5411 or haskell/cabal#4648 would be nicer.

[mpickering]

proto-lens-protoc-0.3.1.1 works for me. Thanks a lot.

[hvr]

@judah are you aware that while packages such as proto-lens-combinators don't run anymore into the cannot-find-proto-exe issue, whereas the package proto-lens-protobuf-types still does?