Should Stackage ignore version bounds?

[snoyberg]

See the relevant Stackage Curator blog post at https://www.stackage.org/blog/2018/01/ignore-version-bounds. This issue is for the official discussion of the topic.

[Gabriella439]

I think this proposal should lead with the problem that this is trying to solve.

For example: part of the proposal mentions lightening the load for maintainers, which is one possible justification. However, I believe that people who are sick of babysitting upper bounds have already removed them from their .cabal files, so I think this wouldn't change much for them.

However, I believe there are other potential benefits, too (such as lightening the workload for Stackage maintainers, which is a legitimate reason). Either way, we should focus on a motivating problem to focus the discussion.

[mikeplus64]

This is also what irks me about Hackage revisions: IMO, package authors should never rely on the invisible hand of the Stackage or Hackage folk to fix or update their dependency bounds. Can package authors "just" be better aware of their dependencies?

To that end, maybe have a standardised report.yaml (or whatever) in sdists that reports on valid combinations of dependencies within bounds specified in the .cabal file, whether the build succeeds and whether test suites pass.

For Stack, stack report or something could generate this for e.g. each of the latest LTS snapshots per major snapshot number/compiler version. It could then play devil's advocate by ignoring version bounds and telling the author when their dependencies can be loosened.

i.e., CI for very lazy people :-)

[snoyberg]

I've left out any kind of specific motivation, as I don't have a specific reason for this change. In fact, I'm not really advocating for it, but merely pointing out the potential positives and negatives, and hoping others will weigh in with the importance of those, and any points I've neglected. I agree though that people should definitely explain specific motivations they have for either supporting this change or not.

@mikeplus64 I agree that soft bounds information would ideally be tracked outside of the cabal file itself, but I go much farther: tracking it outside of the tarball completely, in a separate metadata database intended to be mutable from the beginning and only affecting build plan construction, not builds themselves. Unfortunately, that plan is not being considered upstream, and there's not much we can do at the Stackage layer to make that happen. We should consider such changes out of scope for this proposal, and focus instead on what Stackage itself can actually change.

[gbaz]

One issue that arises on stackage periodically is that a bunch of packages are pulled because some upstream package is updated and all their bounds break. It seems to me that one way to ease stackage maintainer burden without moving entirely to ignoring bounds would be to add an allow-newer stanza in build-constraints.yaml that allowed bounds to be removed for specific packages.

This would reduce the flux in nightlies while still informing authors when they're bumped into that section, which ideally would be a temporary rather than permanent state.

[andrewthad]

My two cents:

• Nightlies: ignore the bounds
• LTS: conform to bounds

Effectively, this means providing some slack to existing packages (ones that provide explicit upper bounds) when a new major release of a dependency comes out. But if they haven't done it by the time a new LTS comes around, they get kicked out.

[DanBurton]

As a curator, what I understand to be the difference in our workflow if we were to ignore upper bounds is:

• no more opening issues asking people to relax their bounds
• more diagnosing build & test failures

The former is busy work, but is easy. The latter is more meaningful work, but requires more brain power.

I think if we ignore upper bounds, we should at least make one improvement to our tooling: add a "build plan changes" to the logs for each package that is built, which informs us of how that package's transitive dependencies have changed since the last published build of that type. I usually find myself taking a little while trying to find out a) which changes in the plan have occurred and b) which dependencies affected this particular package. We already have some output that is meant to help with this, but it can get quite long, which is why I think grouping it by package and putting it in log files would help.

With that change, and possibly some other tweaks the "more diagnosing build & test failures" could be less work.

One downside to ignoring upper bounds is that we lose the implicit automated go-ahead that I'm used to when attempting to lift one of the "stackage upper bounds". We would have to more manually track whether all build failures have been resolved, rather than just "ok stackage-curator says the build plan is valid now".

[DanBurton]

@Gabriel439 it's hard to pinpoint exactly one motivation, but here's how I would summarize a few:

• Making Stackage a more lenient downstream consumer of Hackage, less concerned about the outcome of policies regarding trustee revisions, the "uncurated hackage" proposal, etc.
• Making the outgoing notifications from Stackage less noisy and more meaningful to package maintainers.
• Easing the workload of Stackage curators
• Speeding up Stackage's adoption of new package versions: not waiting for every dependent's maintainer to declare that they are ready, instead letting the code speak for itself.

But I see this discussion less about whether we are able to solve some particular problem, and more about simply weighing the pros and cons of ignoring version bounds and seeing which way the scale tips.

I wish we could fork the state of the world and do a test run to see what happens. As it stands, I don't really feel strongly one way or the other, which causes my personal opinion to tip towards the status quo. I also look forward to hearing others' thoughts on the matter, which may rebalance my own opinion. If we do end up sticking with the status quo, then at least our documentation of reasons for this decision will have improved, which in itself would be a valuable outcome of this discussion.

[gbaz]

My general view is that as a downstream "distro," stackage should be able to make these sorts of changes where it finds it appropriate (lifting bounds, etc). Whether or not stackage adopts a "lift all bounds" policy is only one point on the spectrum. In general, being able to toss in certain specified modifications (like in my suggestion for a allow-newer stanza), could just be a way to reduce friction and churn for curators (ideally in a way that can be semi-automated) and decouple the lockstep with "upstream" (by which i don't especially mean hackage in particular, but even more upstream maintainers of all the packages in stackage).

[DanBurton]

I really like @gbaz's idea of an allow-newer section of build-constraints.yaml. It could be used at maintainer's request or at curator's discretion. We could adapt usage of this section to be light or heavy, temporary or permanent, as our understanding of its impact improves over time. (We'd start with light and temporary.)

[blamario]

For the first year or so of its existence, Stackage was viewed by many as "CI for Hackage."

Count me in this group. My packages follow PVP, and I've come to rely on Stackage to tell me about dependency upgrades. When one comes, before I relax the upper bound I always take care to at least review the dependency's change log to make sure there was no semantic change.

As Gabriel says, should I ever get overwhelmed by this process I could simply remove the upper bounds and hope that the test suites catch all problems. So I see no benefit to this proposal.

EDIT: I forgot to mention another benefit I get from the notifications: it's happened a few times that, after reviewing what's changed in a dependency, I find that nothing was broken but there's new functionality that I can use, or even had to implement myself before. It's not all about guarding against errors!

[bergmark]

Should I ever get overwhelmed by this process I could simply remove the upper bounds and hope that the test suites catch all problems. So I see no benefit to this proposal.

You are currently at a disadvantage if you do this when others use upper bounds for the same package. When we post upper bounds issues we haven't tried to build anything, so any maintainers not using upper bounds won't get a notification immediately. They have to wait until we lift the upper bound (potentially weeks later) and actually perform a build against the new version. This could be mitigated if we, in the upper bounds issue, mention all reverse dependencies, but that would notify a lot of users when there is no action they need to take.

[blamario]

When we post upper bounds issues we haven't tried to build anything, so any maintainers not using upper bounds won't get a notification immediately.

That seems more like a problem with the process. What's stopping you from running a build that ignores the upper bounds and then notifying all maintainers whose tests fail plus all the maintainers who only need to bump their upper bounds? They could be separated into two distinct issues, or N+1 issues.

[bergmark]

It's more of a tooling problem. We would currently need to manually disable all packages with offending upper bounds, build with the new version, report issues, and then re-build with the older version again until everything has been fixed.

[ezyang]

Coming into the conversation from the Cabal/cabal-install side, I want to comment a few things:

Author indication of hard bounds. In the Uncurated Hackage Layer proposal https://github.com/haskell/ecosystem-proposals/pull/6 it was suggested that an x-curation property be added. x-curation isn't an actual concept understood by any tool today, but because it has an x- prefix tools will accept it without barfing. I don't see any reason the Stackage ecosystem cannot drive the standardization of another such field, x-hard-bounds (or your favorite name) which allows authors to signal which bounds are actually, "Yes, this is definitely broken please don't use this version", and which is not understood by any tool but fed into the Stackage curation process. Like x-curation, it's a way for package authors to communicate useful information to potential downstream distros (Hackage, in the case of x-curation, and Stackage, in the case of x-hard-bounds).

cabal-install support for Stackage. I think of this as a deficiency of cabal-install: cabal-install absolutely should have a way of being told, "Trust me, don't use the bounds in the cabal file" and in fact, it already does, by means of the allow-newer and allow-older flags. So in principle, Stackage could build a cabal.config that will work even if there were conflicting bounds simply by toggling allow-newer and allow-older on all packages unconditionally.

Interaction with Hackage revisions. The relationship with Hackage revisions is like trying to have your cake and eat it too. Either you have complete control over the Stackage experience (and thus you absolutely must not automatically pull Hackage revisions), or you suck in fixes from Hackage automatically (and thus the workingness of Stackage depends on what the Hackage trustees do.)

I think in an ideal world, Stackage curators would also be Hackage curators, and so if a revision would in fact break Stackage (hopefully a rare occurrence) they would be in a position to intervene, and then Stackage can deal with a universe where packages sometimes get hotfixed outside of the release cycle. In an adversarial world (where Stackage hates the existence of revisions and demands they didn't exist at all), Stackage should only manually pull in revisions. Depends on the relationship you want :)

Some miscellaneous comments:

This is also what irks me about Hackage revisions: IMO, package authors should never rely on the invisible hand of the Stackage or Hackage folk to fix or update their dependency bounds. Can package authors "just" be better aware of their dependencies?

@mikeplus64 Hackage/Stackage want the ability to fix bounds, for the same reason that you might prefer to use open source software or run your own infrastructure: if your stuff is broken, you don't want it to be the case that your users have to wait for upstream to act before a fix can happen. Now, this issue is far less poignant for Stackage than it is for Hackage, since Stackage can unilaterally drop unresponsive packages from the distribution, whereas Hackage has tried to keep as big a tent as possible (and is more likely to end up in situations where someone has unilaterally broken everyone's depsolved plans, and need an intervention to fix everyone.)

[gbaz]

Stackage curators would also be Hackage curators, and so if a revision would in fact break Stackage (hopefully a rare occurrence) they would be in a position to intervene.

We have at least one joint curator, I believe. I don't know of an analogous list to the hackage trustees list where I can determine all stackage curators. If there's an easy place to see this, I'd love to be directed to it. (And if there isn't, maybe there should be?)

[snoyberg]

This list is available in the document covering Stackage Curator processes: https://github.com/fpco/stackage/blob/master/CURATORS.md. I added the list here 5 days ago, previously it was part of the "apply to be a curator" document.

[DanBurton]

Looks like we've stuck with the status quo for a while after this discussion. Closing the issue as I'm not aware of any intent to make changes at this point.