Closed michaelpj closed 2 years ago
I realised it's worth saying something about why consistency matters.
Some people like to format their codebase with stylish-haskell
or remove all lints with hlint
. Some people like to enforce that these conditions are met. But "is this codebase formatted correctly" also depends on the version of stylish-haskell
you use. So your entire team needs to have the right version installed.
Just being able to write dev-tools: stylish-haskell:exe=2.0.0
or whatever would solve this quite effectively.
To be clear what I meant about the command, I imagine something like cabal dev-install
or cabal install --dev
would do it, but I don't really know enough about how cabal install
works to comment sensibly at the moment.
I think this can be implemented as out-of-cabal-install prototype first to figure out the details.
In general I guess this is fine, yet why don't you use Nix? what if there is some dev-dependency which is not on Hackage, or not written in Haskell at all?
For stan
you need same GHC, yet for HLint you don't want to recompile it with different GHCs (you really don't, it takes so long, that you'll lose interest in the project before it completes). There are details to be figured out.
I do use Nix! However:
haskell.nix
has a bunch of machinery to get "another Haskell package but definitely built against the same boot libs as your project" for precisely this case. It would be nicer to just get this information from cabal
, then haskell.nix
can just DTRT out of the box.For sure, you're still on the hook for non-Haskell dev dependencies, which is why I think this isn't a compelling feature if we're
just talking about convenience of installation. But for me the ghc
dependency issue is really quite annoying, and I think could be solved uniquely well by cabal.
I wouldn't mind compiling multiple versions of hlint
for different projects. It's slow and annoying, sure, but it's much less annoying than having the wrong hlint
. But yes, hlint
is a weak case compared to the tools with ghc
dependencies.
OOI, how would you go about solving this out-of-cabal-install? Don't you need access to the solver etc.?
I think haskell-language-server
is a nice one to consider, because:
I will believe when I see a PoC. And as I see it, it can be done as an external cabal-install-dev-tools
executable.
Side note: I really hope that GitHub discussions will soon be available for everyone. This is not an issue, this is an open-ended discussion.
This is wrong place for this kind of discussions. But we don't have better one.
Happy to close and take it elsewhere, but I do think it's useful to have something for people to find if they have a similar idea and I don't know where they would look except here.
See https://github.com/haskell/cabal/issues/6445#issuecomment-654944587
From what I see, the decision is made, so I'm saying this with the hope to find a better way to stay on top of these discussions and contribute in a timely manner when and where they happen.
FWIW we have extra-packages in the cabal.project file. In principle it'd allow you to do cabal run stylish-haskell
if you had extra-packages: stylish-haskell
. Unfortunately it needs a bit of tuning in the target selector to fully work.
I traced the target selector code and found that if you use fully qualified syntax like
$ cabal run :pkg:stylish-haskell:exe:stylish-haskell
It happily starts building stylish-haskell but fails with
cabal: Unknown executable stylish-haskell in package
stylsh-hskll-0.11.0.0-646a8e66
So I guess we might want to
Make the target selector syntax a bit more ergonomic. I think this has to do with how we represent extra-packages as NamedPackage
. They don't carry component info like the other local package types do and can't give good hints for the target selector resolution.
Figure our why the cabal run
code can't find the exe in the package plan (see matchingPackagesByUnitId
)
Yes, I tried extra-packages
, and it's close to what I want, but @phadej indicated that it's somewhat unsupported, so I'm abandoning that. I also hit the issues you mentioned with accessing the executable component properly, but I assumed that was down to extra-packages
being janky.
However, I've realised that I can do what I want by wrapping cabal v2-install
:
I'm going to try out a little tool based around this idea.
I had a quick look and the selector problems seem to be solved in HEAD and I whipped up a quick patch in #6972 to make extra-packages
work correctly!
I was just asking on reddit about the same feature for Haskell projects, basically what npm
offers via package.json
and its dev-dependencies
field, so I am happy to have found this issue here!
@michaelpj what is your current solution right now?
For the Stack project I am working on, I wanted to have this experience with ormolu
, stan
and hlint
. Ideally with hls
also but I left that one out for now as I am not sure how to approach that.
What I ended up doing (with help from @philderbeast - thanks a lot for the idea and initial implementation!) was defining individual stack-ormolu.yaml
, stack-stan.yaml
and stack-hlint.yaml
files that all specify stack-snapshot.yaml
as resolver -> package.yaml
also specifies it as a resolver, so that way I ensure all of the tools are using the same resolver as my project. This is most important for stan
, but it also gives me reproducibility for hlint and ormolu.
Then, to install any of these tools, e.g. ormolu, I use
stack install ormolu --stack-yaml=stack-ormolu.yaml
which installs it into project-local .bin
directory, since that is specified in stack-ormolu.yaml
via local-bin-path
option.
Now I can use ormolu
as .bin/ormolu
.
From what I understood, similar thing can be achieved for cabal
project by running
cabal install stan -w ghc-8.10.7 --installdir=.bin --install-method=copy --overwrite-policy=always
To make these tools even easier to install and use, I also implemented a run
script that kind of plays the role of scripts
in npm run
/package.json
-> it contains a list of commands that use locally installed tools (ormolu
, ...) in opaque manner (they also install them if they are not installed yet). So using this script, you can just do ./run ormolu
and it will install ormolu if needed and then also run it.
How this looks right now:
Does this count as a kind of an external prototype?
I understand that it takes some time for these tools to build per-project, but I don't mind as I find it very valuable to have this super simple process of just calling ./run stan
or ./run hlint
and it all works out of the box, for any developer on any machine, without them having to know anything about these tools or how to set them up for this specific project. This also makes it very easy for me to have the same situation both locally and on CI.
What I find really valuable with npm
/package.json
is this ability to define simple commands that run in the environment that has dev dependencies loaded into it. There is also npx
tool that comes with npm
that runs allowing dev dependencies directly, for example npx stan
would run stan if it was installed as a dev dependency.
@Martinsos Have you tried using extra-packages with a recent cabal release? I think it does exactly what you are after:
Try putting
extra-packages: ormolu, stan, haskell-language-server
into your cabal.project file. You can then use cabal run ormolu
to run the tools.
@Martinsos Have you tried using extra-packages with a recent cabal release? I think it does exactly what you are after:
Try putting
extra-packages: ormolu, stan, haskell-language-server
into your cabal.project file. You can then use
cabal run ormolu
to run the tools.
Hm this indeed looks like a solution (for cabal), thank you!!
I haven't found much documentation on this, so couple of questions:
with-compiler
in cabal.project ensure that?extra-packages: ormolu, stan
, and what I noticed is that cabal used version 0.0.1.0 of stan and version 2.0.0.0 of ormolu. This is the newest version of stan (but it is about a year old) and pretty old version of ormolu (also about a year old). I would prefer to have the newer version of ormolu instead, but I can't because stan is dragging it down with its version bounds - if I set ormolu to be something newer, e.g. ormolu>=3.0
, cabal can't resolve it. Now, I understand this makes sense when it is libraries and it can be a problem if they expose third library through their API but it is not of the same version, however here we are dealing with executables and in theory there is no reason why one would need to be in conflict with another. I am guessing extra-deps doesn't care about them being executables or libraries and just treats them as normal packages? Is there a way to get it to build newer version of ormolu, since I know that it doesn't matter that stan is using older deps than ormolu?I haven't found much documentation on this
Right, the feature only started to work since I put a small patch in last year. And as such documentation is still quite a bit lacking.
Is there a way to ensure these tools are built with the same version of compiler as is used for the project? Will setting with-compiler in cabal.project ensure that?
Yes, with-compiler will do the right thing.
Is there a way to get it to build newer version of ormolu, since I know that it doesn't matter that stan is using older deps than ormolu?
Great point! The cabal solver does have a notion of solving exes from packages differently. We should make use of that for extra-packages. But for the time being I think it's not going to work unfortunately.
I haven't found much documentation on this
Right, the feature only started to work since I put a small patch in last year. And as such documentation is still quite a bit lacking.
Is there a way to ensure these tools are built with the same version of compiler as is used for the project? Will setting with-compiler in cabal.project ensure that?
Yes, with-compiler will do the right thing.
Is there a way to get it to build newer version of ormolu, since I know that it doesn't matter that stan is using older deps than ormolu?
Great point! The cabal solver does have a notion of solving exes from packages differently. We should make use of that for extra-packages. But for the time being I think it's not going to work unfortunately.
Thanks for putting in the work!
with-compiler sounds great then -> I think the only thing missing is getting cabal solver to understand it is executables. But that is problem only when one of the executables is somewhat older I guess / unmaintained.
Then I guess we can close this in favor of a new "independent goals in extra-packages" ticket. I'll do that in a few days if nobody objects.
@fgaz feel like getting around to the above? :-)
/me vehemently refuses to object
Alright, here it is: #7865
Many build tools have the concept of a "dev dependency". These are dependencies which are managed by the build tool, but which are not used by any of the actual build products. Instead, they are there to provide tools that developers of the project will use.
Why have such things managed by the build tool? Can't users just install them separately? There are a few advantages, such as convenience and consistency between developers, but I think there is one that is very important in our case: the build tool may have information that makes it much easier for it to get the "right" tool. For example, many development tools themselves expect to interact with the language toolchain and build tool. It can therefore be important to pick a tool version that's compatible with the toolchain that you're using.
This is becoming very pertinent in the Haskell tooling ecosystem as more things depend on the
ghc
library. A tool which depends onghc
, likehaskell-language-server
, must be built with a version ofghc
that exactly matches the compiler the user is using for a particular project (which may indeed differ between projects!). As it stands, this creates a lot of work for users and developers. Users must carefully manage their tool installations to make sure they have precisely the right version of the tool installed for the project in question.But we already have a tool that has all the information about what version of
ghc
you are using and how that affects what versions ofhaskell-language-server
you can use: namelycabal
!So much for the motivation, what am I concretely suggesting?
cabal.project
learns a new field:dev-tools
, which is a list of component specifications a labuild-tool-depends
.cabal
command will build all thedev-tools
and install them into some project-local location.I don't think the
dev-tools
build should include constraints from the components in the project - the implicit constraints onghc
,Cabal
etc. should be enough.Partial duplicate of https://github.com/haskell/cabal/issues/5588, but I'm making quite a different case, which I think is much more compelling than just convenience.
Many of the other build tools which have "dev dependencies" also use them for things that have better solutions in
cabal
, e.g. for build tools, test dependencies etc. I think there's still a useful role for dev dependencies that are really for developers only.A short list of things I expect people might want to put into
dev-tools
:haskell-language-server
/ghcide
etc.ormoulu
/stylish-haskell
/brittany
etc.hlint
/stan
(stan
does actually rely on.hie
files and hence depends onghc
!)One could even imagine people who build their Haskell with alternative systems like
shake
might use acabal.project
to get theshake
binaries in the first place.