Add "require-cabal-version" and use this to enable Cabal-1.24 build plans

[mgsloan]

The fix to haskell/cabal#1725 is in Cabal >= 1.24. This means that configure doesn't always need to take as many deps (for buildable: false). In https://github.com/commercialhaskell/stack/issues/2475 , this came up as issues with the solver, because the solver thinks some things aren't deps, that stack thinks are necessary deps.

Currently, the Cabal version in use is something selected by the user not the project, and this can be a source of non-determinism. Currently build plan resolution is independent of Cabal version, and we'd prefer to keep it that way. Why? Because then we have better hope for consistent results for users. Lets say we do vary build-plan based on installed Cabal version. Then, a stack.yaml for Cabal-1.24 could easily break for a user of Cabal-1.22.

How do we solve this? I think I prefer the following approach:

1) Add a require-cabal-version field, similar to require-stack-version. Check installed Cabal version on build.

2) When the version range excludes < 1.24, instead use the build plans with fewer deps. This way we can force consistency.

3) Relevant to #2475, for stack init, this would mean that when the user has cabal-install >= 1.24, it will output a stack.yaml with require-cabal-version: ">= 1.24". If the user is using an earlier version, then a warning will be omitted and no require-cabal-version added.

[Blaisorblade]

I understand you're often talking about cabal-install, right? I understand this solution keeps stack init not 100% deterministic, but the rest is deterministic—users of older cabal-installs will get a stack.yaml with more dependencies. And they'll even be built IIUC?

If the user is using an earlier version, then a warning will be omitted and no require-cabal-version added.

Which warning?

I was also thinking of installing and using a fixed version of cabal-install—ideally, 1.24 (and update stack itself to that), but I can't figure out a convenient workflow for it. I found #2386.

[borsboom]

Just to be clear: this would only effect stack init --solver and stack solver, right? But it would have no effect on building an existing stack.yaml (which uses the internally linked Cabal). The flag name should probably be changed to require-cabal-install-version to avoid confusion between Cabal-the-library and cabal-install. Possibly even require-solver-cabal-install-version to make it totally clear where it applies (although that may be overkill).

[borsboom]

Have Stack fully control the cabal-install version used for the solver (as suggested in #2386) might be a better solution to this non-determinism. Having the user need to self-install cabal-install has always been an annoying part of the workflow. I think we weren't sure at first where things were going to go with the solver and so weren't worried about it being non-deterministic and annoying, but it's clearly an important part of many users' workflows and we should probably make it nicer to use.

[Blaisorblade]

Just to be clear: this would only effect stack init --solver and stack solver, right? But it would have no effect on building an existing stack.yaml (which uses the internally linked Cabal). The flag name should probably be changed to require-cabal-install-version to avoid confusion between Cabal-the-library and cabal-install. Possibly even require-solver-cabal-install-version to make it totally clear where it applies (although that may be overkill).

@borsboom IIUC not quite. According to #2475, the question is whether stack code should be bug-compatible with the old or new behavior of cabal. (Both Cabal-the-library and cabal-install—their major versions are in sync, so that's OK).

Still, sticking to one cabal version is probably simpler.

[borsboom]

In that case, might make sense to upgrade Stack to build with Cabal-1.24 and also always use cabal-install-1.24 for the solver.

[sjakobi]

If I understand #2475 correctly, the Cabal version in question is the one that we use to build packages' Setup.hs (and that one can upgrade with stack setup --upgrade-cabal, not the one that stack depends on as a library.

So I think the question is whether we can impose that Cabal version on the users.

One possible reasons why we cannot: we want users to be able to use the Cabal version that they depend on for their custom Setup.hs.

I assume that there was a design decision made on supporting different Cabal versions at some point but I don't know what arguments were discussed at that time.

[borsboom]

Ahh... ok that changes things. Would it make sense then to fix the Cabal library used for Setup.hs to a specific version in stack.yaml rather than just specifying a minimum? That seems like the most reproducible way. If the version is not in stack.yaml, then always use the version implied by the resolver (whatever Cabal version comes with the GHC version). And then it might also make sense force the cabal-install version used by the solver to match the cabal library version (at least in major version; sometimes the minor versions get out of sync).

On Mon, Aug 15, 2016 at 6:56 AM Simon Jakobi notifications@github.com wrote:

If I understand #2475 https://github.com/commercialhaskell/stack/issues/2475 correctly, the Cabal version in question is the one that we use to build packages' Setup.hs (and that one can upgrade with stack setup --upgrade-cabal, not the one that stack depends on as a library.

So I think the question is whether we can impose that Cabal version on the users.

One possible reasons why we cannot: we want users to be able to use the Cabal version that they depend on for their custom Setup.hs.

I assume that there was a design decision made on supporting different Cabal versions at some point but I don't know what arguments were discussed at that time.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/commercialhaskell/stack/issues/2479#issuecomment-239808002, or mute the thread https://github.com/notifications/unsubscribe-auth/ACEGJQhV6qWNg7398ll0Ow0l3EbUF-u4ks5qgHAdgaJpZM4Jis7m .

[Blaisorblade]

Problem: GHC 7.8.4 would regress to a worse solver. I still have nightmares about getting stack to build with cabal-install 1.18 on Travis, though maybe now it would work (with the new backtracking settings).

On Aug 15, 2016 23:15, "Emanuel Borsboom" notifications@github.com wrote:

Ahh... ok that changes things. Would it make sense then to fix the Cabal library used for Setup.hs to a specific version in stack.yaml rather than just specifying a minimum? That seems like the most reproducible way. If the version is not in stack.yaml, then always use the version implied by the resolver (whatever Cabal version comes with the GHC version). And then it might also make sense force the cabal-install version used by the solver to match the cabal library version (at least in major version; sometimes the minor versions get out of sync).

On Mon, Aug 15, 2016 at 6:56 AM Simon Jakobi notifications@github.com wrote:

If I understand #2475 https://github.com/commercialhaskell/stack/issues/2475 correctly, the Cabal version in question is the one that we use to build packages' Setup.hs (and that one can upgrade with stack setup --upgrade-cabal, not the one that stack depends on as a library.

So I think the question is whether we can impose that Cabal version on the users.

One possible reasons why we cannot: we want users to be able to use the Cabal version that they depend on for their custom Setup.hs.

I assume that there was a design decision made on supporting different Cabal versions at some point but I don't know what arguments were discussed at that time.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/commercialhaskell/stack/issues/2479#issuecomment- 239808002, or mute the thread https://github.com/notifications/unsubscribe-auth/ ACEGJQhV6qWNg7398ll0Ow0l3EbUF-u4ks5qgHAdgaJpZM4Jis7m .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/commercialhaskell/stack/issues/2479#issuecomment-239931479, or mute the thread https://github.com/notifications/unsubscribe-auth/AARsqIEsBwr7taqSeobalVgVxeeAiGPGks5qgNbpgaJpZM4Jis7m .

[mgsloan]

Would it make sense then to fix the Cabal library used for Setup.hs to a specific version in stack.yaml rather than just specifying a minimum? That seems like the most reproducible way.

That would be the most reproducible way. Note that the proposal is to use version ranges, the same way require-stack-version works. Why do I want that? Two reasons come to mind:

1) Cabal is supposed to be largely backwards compatible

2) Currently we install Cabal to the global DB. Not sure if we handle everything perfectly here (does stack setup --upgrade-cabal break things? I haven't had it cause problems), but that way packages in the snapshot can depend on the Cabal version. Perhaps Cabal and all downstream packages should always be local, but then you'd need to rebuild Cabal for every new project. Yuck!

Problem: GHC 7.8.4 would regress to a worse solver

Would it? New Cabal can work with GHC 7.8

[Blaisorblade]

Cabal is supposed to be largely backwards compatible

I understand Cabal 2.0 and later releases won't be (necessarily) so compatible. Everybody is unhappy about existing APIs, and there's major surgery ongoing. See haskell/cabal#3573 and haskell/cabal#3566.

Problem: GHC 7.8.4 would regress to a worse solver Would it? New Cabal can work with GHC 7.8

Sorry for being unclear—I was referring to using "whatever Cabal comes with the GHC version".

BTW, in #2517 I suggest to switch to the latest Cabal 1.24 to handle #2515 (support setup-depends, which is only parsed by Cabal >= 1.24). Unless we want to backport 1.24-specific features to older versions, but I guess we don't want to spend time for that.

However, that's the Cabal stack itself depends on, not for Cabal itself. Then, #2515 itself suggests that Setup.hs might need their own .stack-work, because it might depend on different libraries versions. Specifically, two Setup.hs for packages in the same build plan might require different Cabal versions. (Or at least, setup-depends was designed like that). BTW, if Setup.hs has its own build plan, it might need a way of fixing the plan. (I'd like to claim this is seldom or never needed, but I don't know).

[mgsloan]

True, I suppose one possible approach might be to demand Cabal-1.24. Are there any packages we want to build that demand < 1.24?