Discussion: Better support for development plugins

[sol]

Long story short, what I want is make it possible for a developer who works on a specific Haskell package to customize her development workflow with plugins as she desires. She should be able to do so without propagating her personal choices to everybody who is collaborating with here on the same Haskell package.

My motivating use case is using a custom formatter with Hspec. I assume that a solution to this problem will require the support of the build tool that is used to build and test the Haskell package under development.

I did a short writeup at https://github.com/sol/haskell-plugins#a-collaborative-approach-to-haskell-development-plugins that motivates this story in more detail and proposes a solution.

My goal for this discussion:

1. Establish a general understanding that this is a problem that we want to address; or alternatively identify any differences that we might have and figure out whether these differences are resolvable.
2. Decide on how and where we want to further hash out the design.

I recognize that my proposed solution may not necessarily qualify as particularly "beautiful". However, similar stuff can be done in other language ecosystems and I strongly feel that the Haskell community deserves a solution to this problem as well. As such I welcome constructive feedback and discussions as long as they focus on a suitable solution to the problem.

Once we have a general understanding I will proceed to do a proof-of-concept implementation in tinc. For an implementation in stack I might need some help from you guys.

[snoyberg]

My concern with this is: it's a significant amount of build system complexity, which seemingly is geared towards preventing a few extra lines of code being added at development time. I'm not 100% sold on it being worth attacking, since in general I think our Haskell build system is too complex already. (Example: you could almost certainly achieve everything you've mentioned with some crazy Setup.hs file...)

Exploring this further, it seems like the overriding goal is:

1. Write some kind of a config file
2. Based on that config file, add dependencies
3. Based on that config file, affect the runtime of test suites and benchmarks

Starting at (3): since this is a development-only environment, the config file should still be available to read. It seems like the test suite itself could check for the presence of the config file and change its behavior based on that, with no interaction from the build tool.

For (2), I think we're in a funny position: you want to inject an extra dependency into the generated .cabal file. It seems like you already have full control over allowing that, simply by tweaking the behavior of hpack. Maybe I'm misreading it, but it seems like allowing some directory of config files to override behavior of hpack would address this. It also seems like that could be a cool general solution to a class of problems, such as "I want to be able to share this set of default language extensions across lots of projects."

[sol]

Hey @snoyberg thanks a lot for taking the time to give this a review.

which seemingly is geared towards preventing a few extra lines of code being added at development time

Yes, for a common hspec setup it's a change to package.yaml and test/Spec.hs. My main concern with this is that your working tree is dirty and you have to stash them around all the time.

I think our Haskell build system is too complex already

I certainly see that point.

It seems like the test suite itself could check for the presence of the config file and change its behavior based on that, with no interaction from the build tool.

If hpack injected the required dependencies, then yes, that could work. But let's assume a user enables a custom formatter globally. Now, at the point he hacks on some vanilla .cabal project he will get a build error due to missing dependencies.

It seems like you already have full control over allowing that, simply by tweaking the behavior of hpack.

Hmm, the generated .cabal file would then be unsuitable for Hackage releases (or any other form of distribution). How would we deal with this in the context of cabal sdist / stack upload?

I can understand that you are not too excited about this. Who likes extra complexity after all.

For completeness, I want this feature so that

1. I can document how to write and use custom formatters for hspec
2. I can refer people to that documentation when they are not happy with the default formatter
3. users can experiment with new approaches and am not the bottleneck for new innovations

[snoyberg]

Now, at the point he hacks on some vanilla .cabal project he will get a build error due to missing dependencies.

Is there anything that can be done at the Stack level about editing vanilla cabal files? I believe only a custom Setup.hs will help out there.

Hmm, the generated .cabal file would then be unsuitable for Hackage releases (or any other form of distribution). How would we deal with this in the context of cabal sdist / stack upload?

I must be misunderstanding things then. I thought the modification to the .cabal file would be adding an extra build-depends line, which should be perfectly suitable. I'm clearly misunderstanding this though, sorry.

[sol]

Is there anything that can be done at the Stack level about editing vanilla cabal files? I believe only a custom Setup.hs will help out there.

The idea is that the build tool will transparently add the dependency. It should do so for both .cabal and package.yaml based projects.

I thought the modification to the .cabal file would be adding an extra build-depends line, which should be perfectly suitable.

Yes, the build tool should add a dependency. But it should do so transparently (or maybe temporarily is a better word?!) when reading the file.

Ideally this modification will never hit the disk, but if (say for implementation reasons) it has to be written to disk we would want to restore the original state after we are done.

I imagine that this is the actual challenge when implementing this. I think the other stuff is relatively minor and can easily be tucked away into some library.

[sol]

There is something I could do as some sort of a test balloon: Avoid keeping .cabal files on disk for package.yaml based projects.

This story may already have merits on it's own and it faces the same challenges.

[snoyberg]

I think it's basically impossible to transparently change a .cabal file with the way we use Cabal (calling out to Setup.hs as an executable). We've wanted to do it in the past to work around Cabal bugs. So this my be somewhat moot.