search

haskell / cabal

Official upstream development repository for Cabal and cabal-install
https://haskell.org/cabal
Other
1.62k stars 695 forks source link

Add config flag to control generation of .ghc.environment files #4542

Closed hvr closed 5 years ago

hvr commented 7 years ago

Some people are excited about .ghc.environment files, while others are mildly annoyed. I think there's probable cause to make this configurable on cabal's end.

To this end, I suggest (modulo bikeshed) to implement a flag, e.g. --pkg-environment-scope=LEVEL, where LEVEL can be one of

  1. all (default & current behaviour, generate ghc env files with all transitive dependencies of project's (non-qualified) goals)
  2. build-target (e.g. :pkg:Cabal or lib:Cabal or Cabal:test:parser-tests): generate ghc env files containing only the stated goal's /direct/ dependencies)
  3. - (or none or off or disable?): disable generation of any ghc env files

This flag would also be persistable via cabal.project.(local).

Note: Item 2. is not the short-term goal of this feature request! I've included the 2. item mostly to motivate why it makes sense to design this to be a flag that's more than merely a boolean --{disable,enable}-package-environment-files flag.

Btw, a dual feature-request (allowing to opt-out from interpreting .ghc.environment files from GHC's side) has been filed at GHC #13753 -- however, that one will be too late for GHC 8.0.2 users.

Current status:

/cc @RyanGlScott

ezyang commented 7 years ago

So, I've recently been working in the Python/Anaconda world, where they have scripts like source activate myenv which kick you into an alternate shell where when you call python, you get the environment (interpreter, packages, etc) correspond to the particular environment you are working on. Letting this setting be done on a per shell basis seems better to me than setting it to file path location? (Yes, I know this is totally asking GHC environments to be redone, but it seems to me like it would solve the root cause of the problem you're describing above, as well as other problems.)

hvr commented 7 years ago

Tbh, I have been working with Python's virtualenvs for quite some time, and they were somewhat unsatisfying to me. I prefer the git-style approach which has been copied by many tools which are $CWD sensitive. I don't want to have to remember and having to explicitly perform redundant keystrokes to enter an environment other than simply cding into a project, and have my tool follow the DWIM principle. If we require to explicitly enter environments, we still have cabal exec or cabal repl; but that's a different paradigm (which has its uses) from what GHC environment files provide. Since GHC environments started working in cabal head, I've reduced my use of cabal new-repl to quite a lot less.

This ticket is primarily to help those who don't subscribe to that paradigm, and want to opt-out.

alanz commented 7 years ago

I just spent a significant chunk of time trying to get a project to build and run tests, using stack. I build the project using both cabal master and stack, to check that it works for both.

Having deleted the dist* directories and built with stack, the tests failed, complaining about a missing package db in ./dist-newstyle.

After re-installing everything haskell related, I eventually tracked it down to this file being silently generated, and then being picked up by other tools.

At this stage I would prefer to opt in for this generation, or at least get something in the output to say that the file has been generated, so that I can know that a potential landmine has been planted.

This is bound to trip up others as well, as the existence and use of this file is not well documented.

ezyang commented 7 years ago

I agree that this state of affairs is not great. @hvr, can we perhaps change this setting to be opt-in?

23Skidoo commented 7 years ago

Can we get stack to turn off parsing of .ghc.environment files with -hide-all-packages?

/cc @mgsloan

cocreature commented 7 years ago

fwiw nix also breaks in some cases due to .ghc.environment.* files so I don’t think that a workaround in stack is the right solution here.

gbaz commented 6 years ago

As per https://www.reddit.com/r/haskell/comments/8iyvoo/psa_for_cabal_22_new_users_regarding/ I think that the weight of opinion is that these files should not be auto-generated, but only by opt-in.

If we add this flag as a global flag that can be in ~/.cabal/config and off by default then that would help. Then if people wanted such a config, they could run cabal new-configure --package-environment-scope=ALL.

I think that would improve the situation for a whole bunch of workflows...

hvr commented 6 years ago

@gbaz quite honestly, I'm strongly against defaulting it off, as then nobody would even use what I consider one of the biggest improvements over the old flawed user pkg-db model, since almost nobody would know about it ;-(

Before changing anything, I'd rather want to know exactly what workflows we're talking about that people are having problems with, as I suspect the issues are of a different nature...

Fwiw, there's already patches up at

to address some of the concerns with the UI

gbaz commented 6 years ago

Either way, we should have the flag so it can be changed. Lots of people don't want it on, and at the minimum they should have the option. We can sort out where the actual weight of opinion lies regarding defaults in a broader discussion.

gbaz commented 6 years ago

To expand: my personal belief is that it is currently unexpected behavior for any new-* command to produce any changes to anything outside of dist-newstyle and the store (except for install, which obviously installs), which is what this does. So it violates least-surprise to run e.g. new-test and then suddenly find your invocation of ghci doesn't do what you expected.

RyanGlScott commented 6 years ago

Moreover, this affects more than just ghci. I often invoke runghc ../haskell-ci/make_travis_2.hs from within a project to generate a .travis.yml file, but have it break due to some .ghc.environment file polluting my package database with different packages than the ones that haskell-ci expects. I end up having to explicitly navigate to another directory to avoid .ghc.environment files from breaking the script.

Ironically, despite the stated goal of having GHC commands "just work" regardless of the current working directory, in practice they have the exact opposite effect. I have to be hyper-aware of which directory I'm in before I invoke runghc now, since picking the wrong directory can ruin any Haskell script I may wish to run.

gbaz commented 6 years ago

The opt-out flag in ghc implemented in one of the PRs linked above solves that use case. The main concern with only having that that I would have is if there's a command invoked that invokes ghc for you, and thus where it is sort of a pain to thread that flag to.

hvr commented 6 years ago

@gbaz First off, I consider the current behaviour the intuitive one; if I'm in a project, I expect all invocations to be operating in that context and not some other random pkg environment which has nothing to do with the project I'm in; so I'm definitely not confused at all that ghci throws you in the current CWD -- I'd rather be annoyed if this wasn't the case, and always had to remember doing some magical invocation in order to place myself into that context; and fwiw, that's also how other tools like Git operate; For tools intended to be used by humans on a shell interactively, this is a very common and established UI idiom.

Otoh, programs which invoke ghc for you will have to stop making any assumption about the implicit package environment; same goes for programs which invoke git for you. And I haven't seen anybody making any drama out of tools like Git behaving this way.

Ryan brings up the issue of runghc; Personally I've moved to using runghc in the same style I use ghci and ghc -- expecting it to be CWD sensitive. runghc-based scripting is unfortunately a weakly defined interface which operates in the old paradigm of having a stateful global+user package db in scope, which was rather fragile to use (and which new-build was designed to get rid of). It had a lot of issues already with old-build (it's integration with cabal sandboxes was also extremly sub-optimal; and which package environments elegantly adress), and for new-build we may want to redesign how scripting is done, c.f. https://github.com/haskell/cabal/issues/3843 and/or find other ways to integrate runghc with profiles. And as for haskell-ci; that script is about to drop its original goal of being a project-less single-module runghc-friendly script, as its becoming too cumbersome to maintain such a huge monolithic module that way (besides that shebang incompatiblity on some platforms...) and the need arises to start using libraries which are not GHC boot libs. So for that you'll have to switch instead to something like having a wrapper script make-travis-yml in your $PATH like I have e.g.:

#!/bin/bash
exec cabal --project-file=${HOME}/Haskell/projs/haskell-ci/cabal.project new-run exe:$(basename "$0") -- "$@"
ElvishJerricco commented 6 years ago

@hvr I've personally experienced many cases of users coming to me asking why their project doesn't build, only to find its because of these files that they didn't notice because they begin with .. This even broke brittany for me, a code formatter that doesn't even invoke GHC as a process or require any packaging information, which took me a nontrivial amount of time to figure out. I think people's expectation of GHC is that changes to its behavior are not to be made implicitly. This is the expectation for virtually all compilers, i.e. C compilers require every bit of configuration to be done on the command line. Every time I want to deal with package databases, I expect cabal to be involved.

Furthermore, this isn't even a particularly effective solution to the problem it's supposedly fixing, because these files can and will go stale. You have to run a cabal command anyway to update it, and who would choose to have to keep track of that in their head?

It's safe to say that in my experience, this is extremely unintuitive, and that seems to be the consensus in this thread. Having this be opt-out would still leave me with virtually just as many confused newcomers asking for help.

hvr commented 6 years ago

require every bit of configuration to be done on the command line ... Every time I want to deal with package databases, I expect cabal to be involved.

I honestly doubt you even understand how utterly unusable ghc would be on the shell if we did that (and also how much GHC already doesn't meet your unrealistic ideal), and how frustratingly annoying new-build would be for those that want to use ghc/ghci w/o having to remember to setup env-vars, passing tedious cli flags, or invoking cabal. This would be even worse than the inconsistent situation we had before new-build. We're still in phase of paradigm shift, moving away from the old-build paradigm, into the new-build paradigm. This requires some new approaches, and we're still figuring out how to optimise the UI. But you're effectively suggesting throwing the baby out with the bathwater before it even had a chance, rather than trying to improve the UI to provide a next-gen seamlessly integrated and convenient shell experience for ghc. I'm dogfooding this feature as much as I can, and I'm quite happy with it; and now the time has come that we need more constructive feedback from more users. Therefore I'm more interested to hear how we can explore and improve this new paradigm, while new-build is still prefixed by new-* and can more easily justify radical UI changes.

ElvishJerricco commented 6 years ago

I don't think it's considered a high priority among most people that they be able to invoke ghci rather than cabal new-repl. It doesn't seem like a particularly important ability to me.

Obviously GHC isn't only configurable on the CLI. I understand that it wouldn't be realistic to uphold this standard 100% (in fact it's quite annoying that you can't configure more of the C compiler outside the CLI). But I think it's an important principle to keep in mind, and this seems like a particularly egregious deviation.

All in all, I'm willing to accept that others do consider this valuable, so you'll get no uproar from me if this goes unchanged. But it would be my preference that cabal didn't generate these files implicitly.

If we don't generate them implicitly, perhaps it could be a cabal subcommand to generate them? Users very often look at help text, so having a subcommand makes them very likely to be aware of the existence of this feature.

RyanGlScott commented 6 years ago

To be fair, there is an important semantic difference between invoking cabal new-repl and invoking ghci in the presence of a .ghc.environment file. The former will always rebuild the entirety of the current project in interpreted mode, whereas the latter will simply pop you into a shell with the project already built.

I want to make it clear that I find both features extremely useful, and I'm not advocating that either one of these be removed. On the contrary, I would like to be more explicit about which ones I'm using at a given time. Currently, the latter is quite easy to use by accident, which can have unintended consequences. Luckily, some progress is being made towards making this less likely, which is nice.

As I've stated (and others have stated) before, my preference would be to go from opt-out to opt-in. But that suggestion seems to have fallen on deaf ears, so I'm not going to pursue this much further.

I too would appreciate a dedicated subcommand that simply generates a .ghc.environment file and nothing else. AFAICT, the only way to do so at the moment is through a side effect of a different cabal new- command, which seems suboptimal to me.

hvr commented 6 years ago

appreciate a dedicated subcommand that simply generates a .ghc.environment file and nothing else

In order for a project's generated .ghc.env file to be consistent, you need to have performed a cabal new-build operation; it makes no sense to generate a .ghc.env file w/o having constructed a build-plan and having populated the referenced libraries into the nix-style stores; furthermore, in order to keep it consistent, we need to hook into every action which may result in the build-plan being changed, in order to have it regenerated. So having merely a one-off way to generate these environments for a project would be very bad UI-wise, as you'd run very easily into inconsistent states; once you go package environments, you need to keep doing it.

RyanGlScott commented 6 years ago

OK. In that case, it would be nice if the cabal new-build --help output mentioned that it does this (or better yet, provided a flag which toggles the generation of .ghc.environment files on/off). Currently, this information is not very discoverable (except by accident when things break).

ElvishJerricco commented 6 years ago

How do we reach consensus from here? It seems like we've only got one testimonial in favor of opt-out, vs several in favor of opt-in.

lspitzner commented 6 years ago

@hvr

I honestly doubt you even understand how utterly unusable ghc would be on the shell if we did that, and how frustratingly annoying new-build would be for those that want to use ghc/ghci w/o having to remember to setup env-vars or invoking cabal. This would be even worse than the situation we had before new-build.

Indeed it is clearly possible that I too don't understand this. And I may misunderstand what either of you meant by "configurability on the commandline/of the CLI". Could we just spell things out? What input does GHC(i)'s behaviour depend on?

From the top of my head I can think of

1) commandline args, 2) environment bindings such as PATH, GHC_PACKAGE_PATH, HOME 3) INSTALLPATH/lib/ghc-x.x.x/settings 4) the installation ("global") package db 5) .ghci files in cwd and in $HOME 6) .ghc.environment.* files

Is this list complete? Does a complete version exist (in the ghc user-guide)? I just searched for "ghc.environment" (and also just "environment") and could not find anything, yet this seems highly relevant for general reproducability.

But even if the environment files are just an addition to some long list, it appears to me like most other inputs are either

and in general i'd say 1-5 are all either are explicit or change only when the user (knowingly) fiddles with stuff. An implicit, frequently changing, new input is rather different imo.

@ElvishJerricco

Furthermore, this isn't even a particularly effective solution to the problem it's supposedly fixing, because these files can and will go stale. You have to run a cabal command anyway to update it, and who would choose to have to keep track of that in their head?

@RyanGlScott

To be fair, there is an important semantic difference between invoking cabal new-repl and invoking ghci in the presence of a .ghc.environment file. The former will always rebuild the entirety of the current project in interpreted mode, whereas the latter will simply pop you into a shell with the project already built.

Granted both ghci behaviours are useful, this is hardly a reason to not just have two versions of cabal (new-)repl (or a flag or whatever). This path would avoid implicits, could properly handle the staleness case, at the "cost" of requiring the user to invoke cabal for a task that in my view seems to fall under cabal responsibility, not GHC. So I don't see the downside here, apart from the annoying aspect of having to re-think the UI around this.

(GHC is not supposed to know about projects in any way, and I would not expect it to be "clever" in this direction. cabal also (indirectly) invokes gcc, but I would never expect to be able to call gcc and automagically get any haskell-specific behaviour.)

gbaz commented 6 years ago

How do we reach consensus from here? It seems like we've only got one testimonial in favor of opt-out, vs several in favor of opt-in.

This discussion is very clarifying about the different attitudes involved. I think the basic question is not about cabal per-se but rather how ghc should interact with in-path "config" files of various sorts (currently mainly environment files, but also ghci files, and in the future potentially others). Some people expect ghc to behave the same no matter what directory it is run in, and some people expect it to operate in a context-sensitive fashion.

Ideally we could have a solution that satisfies everybody, and ultimately I think that solution is about ghc primarily, and only secondarily about cabal. It is really just cabal auto-generating these env files that highlights the different expectations about ghc user experience.

My suggestion would be a plan and discussion regarding ghc, perhaps ultimately resulting in a ghc proposal, to clarify the principles of how the compiler interacts with various configuration files. Which is to say, if people who don't like ghc picking up env files could configure ghc to not do that, then the question of cabal generating them or not would not arise.

I have a draft proposal about dot-ghc-files (https://github.com/gbaz/ghc-proposals/blob/patch-2/proposals/0000-dot-ghc-file.rst) that I did not yet submit, in part because I hadn't worked out the interactions with or relationship with env files. I can try to work up different version that takes this discussion into account, and we could proceed from there...

angerman commented 6 years ago

@gbaz

This discussion is very clarifying about the different attitudes involved. I think the basic question is not about cabal per-se but rather how ghc should interact with in-path "config" files of various sorts (currently mainly environment files, but also ghci files, and in the future potentially others). Some people expect ghc to behave the same no matter what directory it is run in, and some people expect it to operate in a context-sensitive fashion.

This has also been the gist, I've taken from this. With a slightly different conclusion though.

I almost always prefer non-magic over magic, and the environment files fall clearly into the magic category. @hvr's GHC patch to make it more explicit that they are used in ghci, improves the situation, but it only addresses the symptoms, not the magic.

I'd much rather see

$ cabal ghci  # ghci with environemnt files 
$ cabal ghc   # ghc  with environment files
$ ghci  # stock ghci, no env files
$ ghc   # stock ghc, no env files.

This would (at least to me) read like:

$ <context> <cmd>

If I want to run ghci in the cabal context, I'd run cabal ghci.

On the implementation side, I'd suggest that ghc learns a flag -use-env-file and cabal ghci would just invoke ghci -use-env-file.

gbaz commented 6 years ago

There already is a -use-env-file flag, named --package-env. Cf the docs.

Having cabal and other build-tools pass this flag, and ghc never auto-pick up anything seems a valid bikeshed color. But so too does a global ghc option as to whether ghc should auto-pick things up, set in a potentially minimal global ghc config. That way everyone can choose their preferred workflow, instead of needing to slug it out over the one "true way" :-)

ElvishJerricco commented 6 years ago

@gbaz I'm not opposed to the idea that this is a GHC problem, and that it be configurable in a global GHC config. But FWIW, I do consider it faulty to autogenerate files that are guaranteed to be made stale once the .cabal file is changed. As is, if you want to make a change in the cabal file, you have to remember to re-run cabal new-build before you can rely on the env file. That's a pretty large surface area of new user-error cases. I suppose that's really a different issue, but the solution to it (@angerman's proposal) solves both that problem and the original problem of confusion caused by unknown autogenerated files. It seems far more elegant that you need to invoke cabal to reach your environment. And as long as cabal-install isn't autogenerating these files, I'm not personally bothered either way by whether GHC picks them up automatically.

Regardless, whatever the implementation detail, is it at least agreed that the consensus (w.r.t. unexpected behavior) is on some form of configurability, favoring opt-in rather than opt-out?

gbaz commented 6 years ago

@ElvishJerricco but all files cabal generates are stale once a file is changed. Everything in dist-newstyle too for example :-)

It seems to me that one proposed path forward would be to A) create cabal ghc and cabal ghci commands that always explicitly pass local config files, B) always generate local config files, C) change ghc to be configurable on if it picks up local config files or not, with either default being fine by me.

One tweak could be to put generated local config files into dist-newstyle instead, since that 1) maintains the idea that unless you install stuff, only dist-newstyle and the store are affected by cabal commands and 2) means that ghc won't see the generated files in its path automatically, since they won't be in the path, typically :-)

Along with the "default flow" question is herbert's suggestion that unless we make this feature the default path, people won't use it, which would be unfortunate. I.e. he is arguing that we get this important feature more visibility by making it the default approach. I'm less sure of that -- in particular, if it is useful enough, I think that people will seek it out. And in general as we roll out new cool features we'll want good documentation and also blog posts explaining and summarizing how to do things. I don't see a way around user education in general.

However, I disagree with @ElvishJerricco on the idea that we can say what a "consensus" is at this point. The representative sample of opinion in this thread is very small, and necessarily biased towards those who have taken an interest in this issue because it has affected their particular workflow. (And this thread is composed of power-users with particular and idiosyncratic workflows, honestly speaking). So a straw poll here really can't help us much with thinking about what the average user mainly will want, imho. My take with kicking this to a ghc rather than cabal configuration question is first that it really is that, and second, we have a general process for trying to solicit wider feedback in such a case, a ghc steering-committee to kick thorny issues to, etc. So process-wise it seems a better setting to resolve this as well.

ElvishJerricco commented 6 years ago

@ElvishJerricco but all files cabal generates are stale once a file is changed. Everything in dist-newstyle too for example :-)

I don't know about that. Is a file really stale if it's guaranteed to be updated the next time you try to use it? For the most part, there's not a lot of reason to use anything inside dist-newstyle unless it's via the cabal command that will update it. The env files, however, are not updated by the tool that uses it.

cocreature commented 6 years ago

My take with kicking this to a ghc rather than cabal configuration question is first that it really is that, and second, we have a general process for trying to solicit wider feedback in such a case, a ghc steering-committee to kick thorny issues to, etc. So process-wise it seems a better setting to resolve this as well.

The problem with that approach is that in the best case we will get this for GHC 8.6 (and it’s certainly possible that the deadline for that will be missed as well) while GHC picks up environment files starting from 8.0.

I also don’t think that a global GHC option really solves this: Let’s say you are in favor of environment files in general, then there is still a good reason to want to stop a build tool from generating them namely the case where you switch between different build tools and want to prevent one of them from generating environment files. This is certainly not a particularly common usecase but it’s one that I use quite heavily and ideally one that would be supported.

Regardless of what we take away in terms of a consensus on a default (fwiw I’d be in favor of opt-in), I think this thread shows pretty clearly that different people prefer different workflows and it seems like an option in cabal-install to control the generation of env files could enable this workflows relatively quickly while we can still debate the default and whether there also should be a way to control this via a GHC setting. So I think it would be a shame if the (afaik uncontroversial) feature of adding an option to cabal-install would be blocked by more widereaching and controversial discussions.

ElvishJerricco commented 6 years ago

@cocreature I don't think there's any debate that it should at least be configurable in cabal-install. The two remaining questions seem to be:

  1. Should the use of env files be opt in or opt out, regardless of how that is achieved? (Almost everyone is saying opt in.)
  2. If opt in, then how do we make it opt in? Via making the cabal-install config (which will exist either way) opt in? Or via making the GHC feature opt in?
23Skidoo commented 6 years ago

The config flag for opting out of this should be easy to implement, patches welcome.

kindaro commented 6 years ago

Note: There may not even be ghci, runghc and other ...ghc... commands in $PATH.

For instance, I installed cabal-install out of stack, and it is no secret that stack does not expose any single version of ghc, but manages them internally. cabal commands somehow manage to live with this, but I can only possibly run cabal ghci or cabal new-repl, never ghci.

Even if we disregard the existence of stack, once there is one way to obtain such unusual installation, I assume there may be other ways as well.

Therefore, we should avoid informing the design of cabal-install by the assumption that ...ghc... commands are directly available to the human operator.

quasicomputational commented 6 years ago

How about this:

I think this would be a strict improvement: when you need to inspect things using the package environment at the command line, you are able to do so reliably; when you don't need to, you are in no danger of being caught out. This is like the 'opt-in' idea from above, but I think it's potentially important to be able to get the package environment from the previous command reliably, which the current situation achieves.

hvr commented 6 years ago

@quasicomputational as far as I'm concerned it would not be an improvement at all for my workflow. I'm pretty happy that new-build generates a ghc env file as a side-effect. If it wouldn't it'd be in fact a strict regression for me if I had to keep to remember to invoke a weird new-environment command... which would be even more inconvenient than using cabal new-repl and would also risk getting stale/invalid everytime new-build or new-repl is invoked. Thus, not really a good option IMO.

@kindaro I'm not convinced that your argument is sound to be honest, as its premise is IMO very artificial and doesn't match the common ways cabal-install is used (and yes, we should definitely disregard the existence of stack for the purpose of this discussion as it bears no relevance to it and would only risk to become a useless distraction): cabal in its default config is designed to pick up any executable called ghc that happens to be in $PATH. If there isn't you have to supply one yourself via the --with-compiler option. But that's not too important, as the main point here is that cabal UI is heavily designed around the idea that ghc et al are directly available to the "operator" and cabal autoconfigures itself based on the discovered environment that's available to the operator; most importantly, cabal does not manage compiler installation -- it's the other way around. This happens to be one of the things that I've always liked about cabal (and dislike strongly of tools which force you to always go via them to manage & access the compiler thus violating concern separation).

andrewthad commented 6 years ago

For me, the current behavior (generating the .ghc.environment.* files by default) is preferable. For context, I develop applications and libraries. The generation of .ghc.environment.* files make the ghc and ghci useful. Without these environment files, these two commands are worthless to me. The only contexts in which I've ever seen them be useful in are:

Environment file make it usable with nearly everything.

bgamari commented 6 years ago

I agree with @andrewthad; I find environment files to be extremely useful and it would be a shame to lose the convenience of having them available as a side-effect of a build.

ElvishJerricco commented 6 years ago

@andrewthad @bgamari What if you could just do cabal ghc or cabal ghci, as @angerman suggests? (EDIT: In fact, it seems cabal new-exec ghc does the trick already) The difference between invoking GHC directly and invoking it through a cabal sub command seems entirely negligible to me.

In fact, I would expect to have to use cabal to get things managed by cabal. To me, the fact that bare GHC commands don't provide bare behavior drastically violates the principle of least surprise. The number of people who have wasted non-negligible amounts of time tracking down issues caused by this is an indicator of the surprise factor. I'll again cite my brittany example; simply running brittany with one of these files present in any parent directory is likely to fail. Perhaps this wouldn't bother me so much if GHC didn't search parent directories...

That said, I'd be happy if there were a way to turn this behavior off (at least manually) in ~/.cabal/config.

23Skidoo commented 6 years ago

In fact, I would expect to have to use cabal to get things managed by cabal. To me, the fact that bare GHC commands don't provide bare behavior drastically violates the principle of least surprise.

I find this argument convincing. What if by default the way to run ghc or ghci in current project's context would be via cabal exec [ghc,ghci] and then there was a setting in ~/.cabal/config to enable implicit environment file generation?

andrewthad commented 6 years ago

@ElvishJerricco It's not just an issue of typing cabal ghci vs ghci. How are flags and arguments supposed to be passed to the cabal ghci you propose? Do I need to put a -- in front of everything or do I have a --ghc-arguments='...' with some escaping rules inside the quotes? Currently, everything that wraps ghc in this way has additional rules for passing flags and arguments that have to be remembered. I've always encountered confusion using stack or cabal to exec ghc. Having one command and one source of documentation where all the flags are in one place is less confusing.

My preference is for cabal to continue generating ghc environment files as it has been and to have ghc/ghci continue to pick them up by default. But, I certainly agree that having more configuration options (at the global level) and more warning messages is going to be helpful no matter which way the defaults go.

ElvishJerricco commented 6 years ago

Placing -- in front of arguments is pretty standard, not just among haskell tooling. It would be very unsurprising.

23Skidoo commented 6 years ago

How are flags and arguments supposed to be passed to the cabal ghci you propose?

I think that if we went that route it'd be something like cabal exec ghci -- -foo -bar --baz because of the current limitations of the command-line parser.

ElvishJerricco commented 6 years ago

@23Skidoo FWIW, exactly what you just typed works today (EDIT: clarification: even after you delete the .ghc.environment file). I would propose having direct subcommands to avoid having to type exec.

23Skidoo commented 6 years ago

Not 100% convinced that saving a few keystrokes is reason enough to add new top-level commands, especially if it's possible to enable implicit environments. I'm +1 on adding support for command aliases though, but that needs a rewrite of the command-line parser.

hvr commented 6 years ago

I would expect to have to use cabal to get things managed by cabal

In that case, you're basically saying that ghc and ghci shall only ever be able to access things in your global package db only. That's not very practical.

ElvishJerricco commented 6 years ago

Or whatever I explicitly ask it to access, yea. What's not practical about that?

hvr commented 6 years ago

Because this is about ergonomics: When you're in a shell and have to type stuff interactively, you don't want to hurt your fingers having to type long-winded invocations repeatedly and fill your terminal with redundant noise. You want DWIM optimised heuristics. And that includes the ability to manage the default environment that a bare invocation of ghci or ghc throw you into in a stateful manner; or to follow the CWD-sensitive paradigm that tooling like Git follows when you're inside a project folder. Otoh, the cases where you want to be explicit is usually when you're scripting because there you are typically in an editor and usually don't mind the extra keystrokes needed, as you write the script once but execute it multiple times and it amortizes. I consider being forced to use cabal exec ghci -- ... terrible ergonomics and bad UX.

ElvishJerricco commented 6 years ago

It's not just about ergonomics. It's also about semantics. Though I agree that ergonomics are important, they almost always take a backseat to semantics IMO. The semantics here are not good IMO. Cabal is injecting itself as a dependency of GHC with this behavior; i.e. you must understand a cabal feature (that's not well documented, or at all obvious, I might add) to understand how to use GHC and why brittany just randomly crashed on you. Cabal is making choices for you about how to use GHC outside of cabal, which IMO means it's leaving its jurisdiction.

If the problem is just how many keys you must press to run GHC the way you like, I'd much rather fix the problem by providing cabal ghc shorthands, perhaps that don't need -- and eagerly pass remaining args to GHC. You still have to add cabal to the line, but that's much more minimal than theses semantic decisions, IMO.

CWD sensitivity makes sense with Git because Git is literally a tool for managing your CWD's state. It makes sense for cabal because the metadata provided by the .cabal file is essential to its operation, and scoped to a particular directory, plus the .cabal file is not a hidden dot file and is an obvious center of truth that the user themselves maintains. It makes sense for a variety of tools for a variety of reasons, but GHC by and large isn't one of them.

hvr commented 6 years ago

I'd point out that GHC natively supports

which of those GHC features would you consider legitimate features of GHC and which should be removed in your opinion (if any)? And also, who should be charge of creating/modifying those files/folders?

ElvishJerricco commented 6 years ago

I'd point out that I deliberately chose the phrasing "by and large," rather than "absolutely." Obviously I know there are even aspects of GHC that can be CWD sensitive (such as default output file locations). My point was much more about the semantics that we experience because of these implicitly generated and accepted environment files.

23Skidoo commented 6 years ago

@hvr has a good point about implicit environments being a logical extension of the user package DB concept to the new-build model. This does make sense once you get used to it.

lspitzner commented 6 years ago

@hvr

which of those GHC features would you consider legitimate features of GHC and which should be removed in your opinion (if any)? And also, who should be charge of creating/modifying those files/folders?

I already gave a detailed analysis of this in my last comment. If you disagree with that analysis or I misunderstand your question, please clarify.

I'll just repeat it to make it extra clear: ghc.environment files are different from other sorts of implicit input because they introduce a new type of non-local state which changes frequently and implicitly (as a side-effect of an invocation whose obvious purpose is not to change ghc behaviour).

The comparison to .git falls flat, because I don't use any tools that non-verbosely, implicitly, as a side effect, change any of the .git implicit inputs. The way I use git, its (implicit) state changes only when I call git or when I manually muck around any relevant files. Or, for the case of CWD, when I use cd/pushd/popd whatever, all of which have a simple, clear purpose that is well documented and really hard to invoke unknowingly.

If you want to save a few keystrokes, please just use an alias that turns "cghc foo bar" into "cabal ghc -- foo bar" or something similar. This serves the same purpose without introducing surprising, hard to debug behaviour.

Otoh, the cases where you want to be explicit is usually when you're scripting because there you are typically in an editor and usually don't mind the extra keystrokes needed, as you write the script once but execute it multiple times and it amortizes.

But you can "script" your commandline experience just like everything else, so this whole argument/differentiation escapes me. I don't like surprises, regardless of context.