Come up with a way to build documentation for a package without rebuilding a lot

[michaelpj]

Describe the feature request At the moment, when cabal builds a package, it decides first whether or not it will build it with documentation, and then it builds the entire package (possibly including documentation) and puts it into a single location in the cabal store. (So: the choice of whether or not to build documentation affects the store hash)

This means that deciding to build a package with documentation when it was not built before means:

1. Rebuilding that package
2. Rebuilding every downstream package (including local ones that depend on it), since we now consider it to be a different package

This is especially painful in the post-hi-haddock world, where we really don't need to rebuild a package in order to build the HTML documentation for it (see discussion here).

There are a few possible solutions:

1. Allow storing the haddock in a separate place in the store from the interface/object files. This would decouple the two, so building the haddock would just build some new things and not affect the existing things (thus preventing cabal from thinking it needs to rebuild dependent packages). See here.
2. Always build haddock for non-local packages. Now that a lot of the work is done during GHC's compilation pipeline, I think it's probably fairly cheap to do the final step of generating the docs. In an ideal world we could also do it in parallel with starting to build the next downstream package. I'm not sure how we would configure this, and maybe it's surprising.

[mpickering]

I think you could write a similar ticket to this for other features which affect the store hash as well. For instance, if you decide you want to build your packages with another way, then you have to also rebuild the entire package and all dependencies. For ways, why could we not reuse the already built .o and .so files which are already built if additionally requesting .p way.

Documentation of packages does depend on the documentation existing for dependent packages as well (as described in #8725).

cabal-install is not designed in a way to track fine-grained information like this, the package is the only kind of thing which ends up in the store and I think tracking things in a build system more like shake would be quite a large project.