Mtl 2.3 Release Prep

[emilypi]

This is release prep for the 2.3 release of mtl.

TODO

---

• [x] Add existing typeclass instances for SelectT
• [x] MonadAccum (including documentation)
• [x] MonadSelect (including documentation)
• [x] Compatibility with transformers-0.6
• [ ] Changelog entries
• [x] Remove all re-exports
• [x] Version narrowing
• [ ] Cut a release

[turion]

I can send a MonadAccum PR if people would want this.

[chessai]

Edited the list to reflect a few new things that need to be done

[chessai]

Confirmed that mtl master can build with transformers-0.6

[jberryman]

Would someone be able to cut a release with what's already in master? It's a shame basic stuff like #67 has been sitting unreleased for years.

[emilypi]

@jberryman we may be able to yeah. I think now that the CLC elections are done and we're resolving procedural issues, the next steps are for @chessai and I to sync on when we have time to address this. I'm leaning towards just cutting a release currently. Stay tuned.

[phadej]

Any update on this? Not having mtl release is holding quite a lot from upgrading to transformers-0.6.

[turion]

Sorry, I'm behind with polishing MonadAccum, but I hope it's not holding up the release? I believe it's not necessary to ship it with the next version. But in case it is, I can speed up.

[phadej]

friendly fortnightly ping

[matobet]

Could we, in theory, cut a 2.3.0 release with the transformers bound bump to let the ecosystem upgrade ASAP and deliver the rest of the checklist items in subsequent 2.3.z versions?

[chessai]

This is already being done. I cut rc1 two days ago, and rc2 is coming tomorrow.

Thanks everyone for the continued push!

On Fri, Nov 12, 2021, 19:53 Martin Beták @.***> wrote:

Could we, in theory, cut a 2.3.0 release with the transformers bound bump to let the ecosystem upgrade ASAP and deliver the rest of the checklist items in subsequent 2.3.z versions?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/haskell/mtl/issues/86#issuecomment-967759342, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEOIX22RSMBNVNT6OKKF7LTULXAHZANCNFSM4YYDTYZA .

[phadej]

@chessai that release candidate doesn't seem to allow transformers-0.6

[chessai]

@chessai that release candidate doesn't seem to allow transformers-0.6

Yeah, that's what rc2 is for. I forgot to bump the bounds before cutting rc1 (though I did already test building with transformers-0.6).

[phadej]

@chessai what a reason to not do https://github.com/haskell/mtl/issues/68 in mtl-2.3. I'm afraid that mtl-3 will never happen.

EDIT: I'll write a patch ASAP if we can get it into mtl-2.3.

[mrkkrp]

Is there any estimate when mtl-2.3 will be released?

[chessai]

Is there any estimate when mtl-2.3 will be released?

https://discourse.haskell.org/t/ann-release-candidate-for-mtl-2-3/3687

[phadej]

friendly ping.

[phadej]

Happy new year ping!

[mrkkrp]

Is there any estimate when mtl-2.3 will be released?

[mrkkrp]

@emilypi Is the project maintained? What is the reason 2.3 got delayed by more than 2 months while @phadej's pings are being ignored?

[chessai]

@emilypi Is the project maintained? What is the reason 2.3 got delayed by more than 2 months while @phadej's pings are being ignored?

The project is not unmaintained, there are two other issues and a pull request where discussion has taken place. I didn't feel the need to reply here after replying there. Perhaps that was misguided on my part. I pinged @phadej and others for thoughts/review on #103, which is the primary blocker of 2.3.

[chessai]

rc4 published on github at https://github.com/haskell/mtl/tree/v2.3-rc4 rc4 published on hackage at https://hackage.haskell.org/package/mtl-2.3/candidate

[andreasabel]

@chessai thanks!
Could the test instructions be updated, maybe just in a comment here. (For rc3, they were published at https://discourse.haskell.org/t/ann-release-candidate-for-mtl-2-3/3687.)

Update: I am using the following cabal.project to test RC4:

packages: .

source-repository-package
  type: git
  location: https://github.com/haskell/mtl.git
  tag: v2.3-rc4

constraints: mtl == 2.3.*
allow-newer:
  *:mtl

[andreasabel]

More often, the cabal.project file looks like this:

packages: .

source-repository-package
  type: git
  location: https://github.com/haskell/mtl.git
  tag: v2.3-rc4

source-repository-package
  type: git
  location: git@github.com:ekmett/exceptions
  tag: 28b9b49a51deb7c1343e003dafb4c69ef79f2e8b

constraints:
  , transformers == 0.6.*
  , mtl == 2.3.*
  , constraints == 0.13.3
  , exceptions == 0.10.4

allow-newer:
  , *:transformers
  , *:mtl
  , *:constraints

[sjakobi]

AFAIK the latest info from the maintainers (@chessai, @emilypi) on the state of the v2.3 release is this comment: https://github.com/haskell/mtl/pull/103#issuecomment-1024522023

If the writing of the migration guide is currently blocking the release, I'd like to suggest that you can also open a dedicated issue where people can ask questions about the migration and share patches and advice.

If there is a different blocker, it would be nice if you could share it, so interested contributors can possibly help resolving it.

[andreasabel]

Rin in die Kartoffeln, raus aus die Kartoffeln. So we are back to RC3? Sigh, then I have to rerun my build-checks for everything...

[sjakobi]

Friendly ping @chessai, @emilypi. :)

It would be great if we could get a release that is compatible with transformers-0.6!

[andreasabel]

I think a reasonable step would be to release RC4 as mtl-2.3. This allows transformers-0.6 and introduces minimal breakage.
RC3 breaks mucho, to the point where it is questionable whether it should be released at all, or whether it should be a major-major release (3.0).

[sjakobi]

For reference: There's a new discussion about the state and future of MTL in https://github.com/haskell/core-libraries-committee/issues/52.

[Bodigrim]

As discussed at https://github.com/haskell/core-libraries-committee/issues/52, @kozross is a new maintainer of mtl. @kozross could you please share your vision and timeline?

[kozross]

@Bodigrim Basically, the proposed task list is my vision as it stands. I'm currently awaiting @tomjaguarpaw removing re-exports, as he offered here, as the compatibility with latest transformers has been checked already.

Furthermore, @emilypi and @ekmett - did you come to a decision regarding lower-bounding GHC at something more sensible? This was another concern of mine, but I've not heard from either of you on it.

[ekmett]

I don't have a current felt sense on the lower bound issue about what releases everyone is using right now. I am planning on moving basically all of my own packages to an 8.6 or so floor to get uniform access to DerivingVia and some other things that rather drastically reduce my code footprint, though, and it is not like old mtl will cease to exist for users below that waterline. Above that I don't know.

[kozross]

@ekmett For what it's worth, I consider 8.6 my absolute minimum, so I'd be happy to do the same for mtl. The main reason I haven't moved ahead with anything is because @emilypi wanted your take.

[tomjaguarpaw]

Hello, sorry, I didn't realise that mtl was waiting for action from me. Sorry if my inaction has been holding things up.

To be clear, my offer was not so much to "remove re-exports", but rather "decide what to do with re-exports". Part of what I would have done, were it up to me, would be to immediately release a minor version with a warning that the exports are due to be removed (see https://github.com/haskell/mtl/pull/106). My hope with that would be spread warning of the breakage slightly more widely. I understand from the discussion on the PR, @kozross, you've decided against that course of action.

With regard to actually removing the re-exports, I believe that simply involves reverting https://github.com/haskell/mtl/commit/ecb525a5ba3521d0059eb2396a376b2e91557cff. or equivalently, reinstating https://github.com/haskell/mtl/commit/9e1582a0b5155a0d2072f020773b25b69462bec7, so I don't see that there's anything for me to do.

[ekmett]

I think I'd be comfortable with an 8.6 floor. If you need to go below that line, you generally need to use lots of old versions of things anyways.

[kozross]

@tomjaguarpaw OK, thanks. Sorry about the confusion.

[phadej]

FYI, GHC lower-bound can be quite high: E.g. ghc-9.0 and ghc-9.2 packages depend on exceptions which depend on mtl, so on those GHCs mtl is essentially non-upgradeable (for any complicated install plan which includes e.g. ghc plugins).

OTOH, current GHC mtl master build down to GHC-7.0, so if there is no super urge to clean up all the internal CPPs etc. right now, just don't. transformers-0.6.0.4 builds with GHC-7.0, so I kind of would expect to have mtl compatible with that transformers. (old enough GHCs don't bundle mtl, so using newest mtl is simple).

(IMO, transformers-0.6 should have dropped pre-GHC-8.6 support, as it introduces QuantifiedConstraints conditionally on GHC-version, but that ship has sailed).

[kozross]

@emilypi Vis-a-vis removal of re-exports, which of these are we removing:

1. Specifically Control.Monad, Control.Monad.Fix, Data.Monoid;
2. All base re-exports;
3. All re-exports which are not our own modules; or
4. Something else?

From what I can tell of the reversions, the answer is 1, but I wanted to be sure, since the checklist you wrote doesn't specify.

@phadej The whole point of raising the floor of the GHC versions we rely upon is not to have to cart around CPP cruft, or rather, to cart around less of it. From what you're saying, it seems to be that you believe this cruft can never go because GHC somehow depends on it. I don't quite understand how this works: the boot library version of mtl is, and indeed, should be, pinned to the older version with the necessary CPP cruft to make it function, and newer GHCs should use the updated version, in which said cruft is not necessary. Am I missing something here?

[phadej]

@kozross you misunderstood me.

The most CPP in mtl is on transformers (CPS monad instances), with few __GLASGOW_HASKELL__ >= 707 (for MINIMAL pragma) and MIN_VERSION_base(4,8,0) (AMP). So you can remove the most CPP by raising the floor of transformers version, but still still supporting quite old GHCs (e.g, GHC-8.0+).

CLC has been quite conservative recently. I'd expect mtl future to be conservative as well. (supporting old GHC is not wrong, re-exporting stuff from Data.Monoid is).

[kozross]

I'm not opposed to raising the floor on the required transformers version, but this was never under discussion as far as I understand it. @emilypi and @chessai - thoughts?

I refuse to be bound by the past when it comes to improvements. Nothing is stopping anyone from using older versions of mtl for reasons of continuity, but holding improvements hostage to staying on GHC versions which are frankly archaic at this point is not something I as a maintainer plan to be held to.

[emilypi]

I support raising all floors and stopping this stupid backcompat dick-measuring contest. Please submit the PR's so I can approve them and merge them.

[chessai]

I support raising all floors and stopping this stupid backcompat dick-measuring contest. Please submit the PR's so I can approve them and merge them.

Agree

[kozross]

@emilypi and @chessai - I've sent #108 dealing with some of the points listed in this issue. It's a necessary preliminary to other work - once this lands, I'll tackle the rest.

[andreasabel]

mtl is core in the Haskell ecosystem so I expect it to be inclusive.
Ubuntu LTS 18.04 is shipping GHC 8.0 (https://repology.org/project/ghc/versions), so that would be the maximum baseline.

The immediate goal would be to release a mtl that allows to proceed to transformers-0.6 without cutting off the majority of GHC versions.

It has been argued that support for older GHCs can be achieved with older versions of mtl. Note that this isn't so easy with stackage's one-version policy. As soon as nightly switches to mtl-2.3, so must you.

One needs to balance the burden of maintaining conditionals in a library with the introduction of conditionals downstream. Throwing away information in one place upstream with require engineering equivalent information in many places downstream.

[kozross]

@andreasabel With all due respect to both Ubuntu and Stack, I simply cannot see why I should carry around their legacy, and frankly bad, decisions. GHC 8.0 is at this point something like five years old, and predates a tonne of improvements which the vast majority of the ecosystem can, and does, take for granted. Likewise, why anyone would use Stack in 2022 is completely beyond me, as literally any real benefit it ever had has long been superceded by a combination of cabal-install-3.0 and above and ghcup, even on Windows, which until fairly recently had been the only real use case.

Limiting to GHC 8.6 is not 'cutting off the majority of GHC versions' - it is cutting off a small minority, even by the most conservative estimate. It also allows us to rid ourselves a lot of maintenance cruft, which frankly should not be our responsibility to cart around because of (ultimately) downstream decisions. If Ubuntu want to package a five-year-old compiler, it's their problem to work with our decisions, not the other way around. Likewise, if Stack have a 'if we move so must you' policy of the sort you describe, it's their bad decision, and people should blame them and work around their bad choices, rather than forcing us to do it instead.

Last but not least, absolutely nothing is stopping absolutely anyone from just using the older mtl for compatibility: the older versions will not vanish magically just because 2.3 comes into existence. Them choosing to do this limits them way beyond something like transformers-0.6, but that's a problem that's pretty much outside of the purview of mtl. I simply do not accept this idea that I have to cart around the lowest common denominator of assumptions for every downstream possible forever and impede progress and improvements on that basis.

[tomjaguarpaw]

I'm quite puzzled all around here.

On the one hand, it won't hurt me personally if mtl's GHC dependency is raised all the way up to 8.6. On the other hand, making a reasonable effort to maintain a wider range of supported versions seems desirable. Perhaps the effort required is unreasonable, though.

Now, @andreasabel points out that once Stackage nightly switches to mt-2.3, anyone using nightly must also switch to mtl-2.3. But this is puzzling to me for a couple of reasons. Firstly, someone who wants to not upgrade could just not be on nightly, couldn't they? Secondly, doesn't using a particular Stackage package set require you to use a particular GHC version anyway? Can someone on nightly actually use GHC < 8.6 at all? (I'm not a Stackage or Stack user so maybe my puzzlement is due to missing some fundamental understanding.)

That said, I'm further puzzled why we can't release a non-breaking version with support for transformers-0.6. If that is a possible option then it seems unreasonable not to take it.

[emilypi]

On the other hand, making a reasonable effort to maintain a wider range of supported versions seems desirable.

Keyword reasonable here. A reasonable effort would be to maintain a fixed window of support for older GHCs. Right now, the baseline of 8.6 is 3 years old. This is a fine window for support. What is unreasonable, is to take on the burden of supporting GHCs that people do not use. The 8.6 baseline is statistically supported by the State of the Haskell survey. 6 months ago GHC <8.6 was already a diminishing ROI. Today, the returns are even less significant. The argument for older GHCs is poor and based upon speculation, and places a maintenance burden on us that has burnt out generations of CLC members in the past across many different packages.

Now, @andreasabel points out that once Stackage nightly switches to mt-2.3, anyone using nightly must also switch to mtl-2.3. But this is puzzling to me for a couple of reasons. Firstly, someone who wants to not upgrade could just not be on nightly

Yes. You opt in to using nightly. And yes, it will be a large coordinated effort ala the recent aeson-2, hashable, text-2, bytestring-0.11 upgrades etc. If people do not want to upgrade, they can pin an LTS, or they can require an upper bound in their cabal file. These are opt-in changes.

That said, I'm further puzzled why we can't release a non-breaking version with support for transformers-0.6. If that is a possible option then it seems unreasonable not to take it.

We've had this discussion, and the maintainers agreed that it would be best to just rip the bandaid off and improve the library instead of constantly kicking the ball down the road. At some point we have to stop being avoidant and accept the pain. We've decided, using our our discretion as maintainers, that this was the time.

[emilypi]

Also, please keep in mind that while Ubuntu LTS 18.04 is pinned to 8.0 in universe, this does not stop the user from installing their own locally, as they would with ghcup or stack. The reason they are pinned to that particular GHC is a mystery, as Ubuntu 20.04 jumps to 8.8 as a baseline in universe. My assumption is that whoever maintained the package stopped, for whatever reason. Either way, I'm not gonna maintain an extra 2 years of GHCs just because people don't want to add these lines to their setup:

> curl --proto '=https' --tlsv1.2 -sSf https://get-ghcup.haskell.org | sh
> ghcup install ghc 8.8.4

Rust has to do this on 18.04, and they've clearly not had trouble with success.

[tomjaguarpaw]

the maintainers agreed that it would be best to just rip the bandaid off and improve the library

Sure, I'm in favour of ripping off the bandaid ASAP. I just don't understand why, at the same time as ripping off the bandaid and releasing 2.3, there can't also be a release of 2.2 that bumps transformers.

[Kleidukos]

@tomjaguarpaw From reading the comments of the various parties here, I can judge that efforts are spread too thin, and such a compatible release would require more time and energy than is currently available.

[emilypi]

It would involve not merging any real fixes for any outstanding issues because they're hard, and just merging more CPP that we'd rip out in the next merge from @kozross the following day. That's not an exaggeration. It would literally be upon @kozross merging master into his branch. Again, we kick the can down the road. Please trust that we've made our decision, and we're going through with it, and we're going to stop derailing for conveniences. Enough is enough.