search

haskell / core-libraries-committee

95 stars 16 forks source link

Decide on core principles against which to judge breaking changes #12

Closed santiweight closed 2 years ago

santiweight commented 3 years ago

Let's decide on some principles that guide the design of base and other CLC-maintained libraries, and let's write them down in a central location.

As it stands, well-thought-out and considered proposals such as Joachim Breitner's no /= in Eq proposal get accidentally hijacked by high-level discussion behind what is sufficient breakage to make a proposal untenable.

For example, within that thread alone, the proposal maker and myself both disagree on the cost of such a change. Joachim argues that the change will cause minimal issues, whereas I argue that while the breakage is relatively small, the change will cause confusion and frustration, especially for those less experienced Haskellers and time-limited library maintainers.

Similar discussion occurred around what should a typeclass' methods contain in #3. In particular I see discussion around whether a typeclass should contain methods. For example, many of the typeclasses contain default functions that are included for the sake of runtime efficiency. However, that principle seems nebulous and not clearly defined. For example, elem and maximum, which require an Eq and Ord constraint respectively, are user-definable in the Foldable class. However, ^^ and ^, which I would imagine can have their own efficient implementations, are not defined on any of their required classes, and therefore cannot be more-speedily defined...

Just to reiterate, this is not specific to any singular function or class (perhaps ^^ and ^ are actually consistent!). The point of this proposal is that we don't have a document or central discussion, as far as I know, that outlines the parameters that make a proposal worth or not worth accepting. If we don't have a central discussion of such simple principles, work is very hard to get done because everyone has a slightly different perspective on what the right balance of breakage-for-improvement vs backwards-compatibility is.

tomjaguarpaw commented 3 years ago

I strongly support this. I will have my own personal opinion about the (/=) proposal but I could be swayed easily to change it if the decision was being made in the context of an overarching design goal for base. At the moment there are good arguments either way and I only have a gut feeling to go on (I believe the same applies to others as well). On the other hand, a more coherent plan for base could make it easy to decide what is the consistent thing to do here.

santiweight commented 3 years ago

Yes exactly - currently we have well-meaning people with thoughtful but differing opinions that cannot be resolved. That is exactly what guidelines aim to prevent!

Bodigrim commented 3 years ago

we don't have a document or central discussion, as far as I know, that outlines the parameters that make a proposal worth or not worth accepting

The README states such parameter: "The committee makes decisions by simple majority voting".

Yes exactly - currently we have well-meaning people with thoughtful but differing opinions that cannot be resolved. That is exactly what guidelines aim to prevent!

I'm sorry, but I strongly believe that this is a rabbit hole of getting nothing done. "Well-meaning people will have thoughtful but differing opinions" about proposed guidelines (and their application) as well, asking for higher level principles and ad infinitum.

It's untrue that differing opinions about base cannot be resolved: the primary purpose of CLC is exactly to provide such resolutions by voting. Indeed, some people will disagree with CLC decisions, but this is life.

@santiweight @tomjaguarpaw you are most welcome to come up with a set of guidelines and their implications for base and present the latter as your proposal (and the former as its justification).

santiweight commented 3 years ago

I'm sorry but I don't buy this at all... Understand, of course, that I value your work and contribution!

What if I want to say, rewrite the Monad of no return proposal. What am I meant to do? Am I meant to just bring it up casually? If I bring it up casually, then I'll likely be shut down for "not being concrete - provide real examples", and rightly so. So what are my alternatives? Spend say 5-10 hours writing up a well thought out and considered proposal only to be shut down by a 10-0 vote because it's not even up for consideration...

But worse yet, I get the impression from the way you're discussing this that there is no plan for base. The whole reason we are here is because no one had a concrete plan for what base would be. It seems that base has incrementally had kruff tacked on, with no clear way to release breaking changes. How often are we allowed to release backwards-incompatible changes? Once every year? Every two years? Once every 6 months with the GHC release?

What about namespace changes? Can they happen any time? This is crucial information for not just a proposal but for base users and maintainers. As it stands, base has remained somewhat stale due to changes that are some mixture of very disruptive but also non-committal enough.

base was already maintained by popular majority in a committee afaik, and the situation hasn't improved. head and last still exist in base to popular derision, and the namespaces for the base libraries are confusing, with much standard functionality (liftA2, join etc.). I don't see how this will ever change unless there is some basic understanding of what base's role is in the Haskell ecosystem, even if the decision ends up being to leave base completely as is...

I'm sorry, but I strongly believe that this is a rabbit hole of getting nothing done. "Well-meaning people will have thoughtful but differing opinions" about proposed guidelines (and their application) as well, asking for higher level principles and ad infinitum.

Yes but they are voted on once. If there are no voted-on guidelines, then we will persistently rehash the same backwards compatibility/extreme breakage discussions and never be able to dismiss arguments as "not within base's scope". No one is asking for higher level principles, just the basics, such as what proposals would even be considered.

I'll cook up some guidelines to discuss, but I think the most important thing here is to have some values and process, even if they are unilaterally constructed by the CLC. Decision of almost any kind would be better than a lack of principles.

Btw, for related work, see the "GHC evolution principles" proposal that Richard Eisenberg posted recently. The specific intention being:

This PR is an attempt to create a central place in the proposals repo for principles. We can then use these principles to guide our responses to individual proposals.

Bodigrim commented 3 years ago

How often are we allowed to release backwards-incompatible changes? Once every year? Every two years? Once every 6 months with the GHC release?

Usually each release of GHC is accompanied with a major release of base, which incorporates backwards-incompatible changes.

What about namespace changes? Can they happen any time?

I'm not sure what you mean by "namespace changes". Potentially they can happen in every major release of base.

What if I want to say, rewrite the Monad of no return proposal. What am I meant to do?

No one is asking for higher level principles, just the basics, such as what proposals would even be considered.

Please refer to https://github.com/haskell/core-libraries-committee/blob/main/PROPOSALS.md for a guidance on how to raise a proposal and what is in scope.

santiweight commented 3 years ago

Thank you for the response :) I am not clear that there is any place where any of this information is outlined. I am only aware of the fact that base is released every 6-ish months because of enough time in the Haskell community to acquire the information via osmosis. Is there somewhere where a newcomer could discover this information?

You say that each release of GHC (which I believe to be every ~6 months) is accompanied by a release of base which might have backwards-incompatible changes. How often are minor changes released?

I am confused - the proposals readme doesn't say anything about what is in scope according to my reading (I've read it through a few times to be sure). I'm not trying to be facetious, but how do I know whether, for example, exporting transformers in Prelude is a potential option? Should I create an issue and have a CLC member respond? The closest thing I see is the following quote:

If you've got insight into how to improve performance, behaviour, or structure of base, awesome! This is the stuff we'd love to hear about. You should follow the steps below, keeping in mind that the bigger the scope of your proposal, the more detailed it should be.

Is your point that any change related to the core libraries is a reasonable proposal?

Bodigrim commented 3 years ago

How often are minor changes released?

Major releases of base are tied to major releases of GHC, minor releases of base are tied to minor releases of GHC.

I'm not trying to be facetious, but how do I know whether, for example, exporting transformers in Prelude is a potential option?

Such proposal would be in scope for CLC.

Is your point that any change related to the core libraries is a reasonable proposal?

The paragraph you just quoted gives an exact description of what is in scope: pretty much any non-trivial change to base. Changes to core libraries (as opposed to base) are normally not in scope, please check https://github.com/haskell/core-libraries-committee#core-libraries for more details on this.

Icelandjack commented 3 years ago

How about a documentation principle, to include counter-examples for type classes: https://blog.functorial.com/posts/2015-12-06-Counterexamples.html

Bodigrim commented 3 years ago

@Icelandjack documentation is generally out of scope for CLC.

chshersh commented 3 years ago

(Not a CLC member)

I'd like to share my view on the subject as a maintainer of multiple open-source Haskell packages. I don't represent any group, this is just my vision and personal opinion. Maybe some can empathise.

I understand the existing discussion as follows (correct me if I'm wrong):

I think Scope is fine. It makes sense to me to have a separate group of experienced people to consider all changes to the standard library as they affect all Haskell developers.

However, I have some concerns regarding Outcome. I think this approach is flawed. It means that the result depends not on some objective guiding principles but on the personal preferences of currently elected members. It's not clear what are the preferences of those people (you can deduce some logic from previous decisions or from knowing such people but it's a lot of work and different people can infer different views). In other words, if you open a proposal at a different time with different members, you might get better luck to see your proposal accepted.

Another point that concerns me is that the existing approach puts CLC members in a position of uncontrolled power. They (in theory) can accept or reject changes based on their mood, personal preferences, or maybe even some personal qualities of the person who opens a proposal. I'm not at any moment saying that CLC has people with such harmful views. But why allow such a possibly dangerous situation to occur if it's extremely easy to prevent this from happening at all? Isn't "making illegal states unrepresentable" one of the Haskell mottos?

Therefore, I believe, it's extremely important to write down explicitly the goals behind the base roadmap.

On a different note, I would like to express my personal views on breaking changes and breaking backwards compatibility.

I think that with the current state of the Haskell ecosystem breaking changes must not be allowed unless:

or

The main goal of the Haskell Foundation is to "broaden the adoption of Haskell". CLC is affiliated with HF. From the website:

Affiliation means that the group supports the goals of the Haskell Foundation, and that the Haskell Foundation in turn supports this group.

In my personal view, introducing breaking changes decreases the adoption. So I don't understand how much this affiliation means in practice.

More importantly to me, almost all people maintain Haskell libraries for free in their free time. By accepting breaking changes you ask volunteers to do even more work if they want to use newer GHC versions on top of their generous contribution to the ecosystem.

I would like to emphasise the following:

⚠️ The size of breakage doesn't matter. Breakage is breakage. ⚠️

When maintainers constantly need to fix tedious issues for no apparent reason, they can easily burn out, leave the community and stop improving the ecosystem.

Again, if base goals allow such breakages then it's fine (I don't think it's a good goal though) but at least this needs to be written explicitly. Something like "the main goal of proposals to base is to have the perfect ideal Haskell standard library, no matter how much breakages it will require". So at least Haskell maintainers are aware of the explicit risks they are taking when they decide to maintain a Haskell library.

TristanCacqueray commented 3 years ago

Would it be possible to require that a breaking change includes a migration strategy and patches for the main projects that are affected?

merijn commented 3 years ago

Major releases of base are tied to major releases of GHC, minor releases of base are tied to minor releases of GHC.

This is not true. This has been the case for the past few years, yes. But mostly due to happenstance. Everytime I bring up tying base version to GHC in the way you propose in the #ghc channel, I've been informed this is not desirable because base (in principle) should evolve separately from GHC and this has happened in the past.

ocharles commented 3 years ago

base (in principle) should evolve separately from GHC and this has happened in the past.

But base can't practically make any release because it's bundled with GHC, and not reinstallable. So holding this stance "in theory" isn't doing anything useful.

gbaz commented 3 years ago

There is at least one guiding policy for breaking changes, which is the three release policy: https://gitlab.haskell.org/haskell/prime/-/wikis/libraries/3-release-policy

"Changes to basic libraries are planned and implemented so that, at any time, it is possible to write code that works with the latest three releases of GHC and base, without resorting to CPP, and without causing warnings even when compiled with -Wall. Such code may not necessarily be idiomatic, though."

This is effectively what's been requested in that breaking changes include a migration strategy.

The three release policy has been a key part of all CLC decisions since its formation (just about), and it may be worth placing it directly in this repo, or highlighting it more explicitly.

goldfirere commented 3 years ago

Context setting: I am not a member of the CLC. Though I am a member of the Haskell Foundation board, I write purely in my personal (and professional) capacity as an interested Haskeller.

I continue to support the proposal suggesting that broad guidelines be articulated for the evolution of base. I think such guidelines serve many goals:

However, I do not think that such guidelines will remove the (in my opinion) essential human element: the CLC is composed of individuals with strengths and weaknesses, and these individuals will vote according to inscrutable internal processes (as we all do in all of our decision making). This means that @chshersh's very valid concerns that the results of a proposal may depend on, say, the timing of the proposal will not be negated by the introduction of guidelines. The guidelines will help, to be sure -- and I support writing them! -- but they will not "fix" this problem.

Instead, I think the best we can do is to make the selection process of these individuals as transparent as possible. The repo currently has no guidelines about how the CLC membership is selected and how it is refreshed over time. I think this is an oversight that should be corrected.

About Haskell Foundation affiliation: I see that the CLC is affiliated (see https://haskell.foundation/affiliates/), but I do not see that the CLC has met all the requirements for affiliation, as listed at https://haskell.foundation/affiliates/about/. In particular, I see no code of conduct or information about refreshing CLC membership. (Maybe there are other gaps, too -- I have not checked closely.) It looks like there is some work to do here.

Timing: The CLC reboot is still very fresh. The housekeeping details I'm advocating for here (guidelines for evolution, rules for membership, code of conduct, etc.) are important, but perhaps not as exciting as actually improving base. Furthermore, I think that some of these details may be addressed better after the current CLC membership has some time to get to know one another and establish some working practices (that could then get written down). I, personally, am thus fine if the CLC wants to delay handling these matters -- until a certain prescribed date, at which point it would return. I really don't want to saddle our new, intrepid CLC volunteers with drudge-work. I also don't want the important governance work completely forgotten.

Thanks, all, for a great conversation here -- it's wonderful to have a place to discuss these important issues, and I am thus very grateful for the work in rebooting the CLC.

szabi commented 3 years ago

In other words, if you open a proposal at a different time with different members, you might get better luck to see your proposal accepted.

As long as the committee's membership is not too volatile, I see no problem in that. A CLC decision once taken can (and should be able to) be overturned later. There should be no ban on reconsidering a proposal, say, 3 years later, as the basis -- the state of base -- has changed by then. There might be merit to the proposal then, even if at its first proposition it was found not to be convincing.

Of course, reopening a proposal, say, 6 months after it was rejected should likely be not considered, as it's unlikely that the fundamental basis has changed enough to merit a different outcome.

sjakobi commented 3 years ago

@chshersh As a fellow maintainer I do empathise with your perspective. I suspect that, as a maintainer of an alternative prelude, you are particularly exposed to breaking changes in base. Fortunately, most packages are not alternative preludes.

While I do feel that breaking changes need to be done carefully, I'm already quite happy that proposals involving them require an impact analysis:

If your proposal includes breaking changes, you must include an impact analysis. How many packages, approximately, will need to be updated? What is your plan for smoothing over that process, and how long do you estimate it will take?

I remember one proposal a few years ago that was approved without an impact analysis, which suggested to remove the Data.Semigroup.First and Data.Semigroup.Last types. In this case, I realized that this would cause quite a bit of trouble once I started implementing it. If the proposal process at that time had involved an impact analysis, I'm sure this would have prevented me from the trouble of getting the decision reverted.

Apart from that, my experience with breaking changes in base has largely been a good one. Changes like the Monoid-Semigroup-proposal, MonadFail-proposal etc. have really improved the language, and I don't remember having much trouble implementing them. But base still contains a lot of historical baggage that will require breaking changes to clean up. I believe that Haskell will be adopted by more people than it has had users so far. It's primarily for these new adopters that base needs to be improved.

I'm very optimistic that the new CLC and the new proposal process will lead to these necessary improvements, and that the CLC and the Haskell ecosystem can manage the breaking changes involved.

Bodigrim commented 3 years ago

⚠️ The size of breakage doesn't matter. Breakage is breakage. ⚠️

The thing is however that CLC is responsible for minority of breakages. Almost every release of GHC intricately breaks type checking. Almost every version of GHC changes ghc-prim in subtle ways and/or template-haskell in arcane ways. Plus overhauls of GHC API. So while we are already on a treadmill, let's grasp this opportunity to do something useful. You see, GHC regularly imposes new challenges for maintainers, but we are still shy to add foldl' to Prelude, because it is a dreadful breaking change.

I'm happy to embrace a principle saying that if there are no breaking changes in GHC, template-haskell or dependencies of base, there should be no breaking changes to base itself.

Ericson2314 commented 3 years ago

As the risk of being a broken record, the essential step is to decouple base from GHC. The total amount of breakage is not an issue, it's that it comes lockstep that makes it's more annoying than it need be, and prevents us from making base actually nice.

juhp commented 3 years ago

What about base5? I mean I had been anticipating that base-5 would to become the smaller, cleaner, modern standard library with some significant breaking changes, also consolidating parts of some popular alternative preludes, and other cleanups etc. ((Showing my age here but I can barely remember base-3.)) I guess there is revolutionary vs evolutionary long term change. Is it practical to start collective work on base-5, or realistic? Obviously the goal of base-5 shouldn't be to break as many packages as possible, on the other hand some people could be more open to one-time major changes rather than continual churn in base-4? Or is it preferable to just evolve base-4 gradually for continuity. From the point of view of really cleaning up the Prelude, base-5 seems necessary at least, but it could take some years to be ready.

ocramz commented 3 years ago

As the risk of being a broken record, the essential step is to decouple base from GHC. The total amount of breakage is not an issue, it's that it comes lockstep that makes it's more annoying than it need be, and prevents us from making base actually nice.

@Ericson2314 Are there any concrete proposals on how to decouple base from GHC releases ?

I'm happy to embrace a principle saying that if there are no breaking changes in GHC, template-haskell or dependencies of base, there should be no breaking changes to base itself.

This ^ seems to be the only practical proposal so far.

Beyond that and documenting thoroughly all breaking changes, I think finding overarching principles for the evolution of base is mostly a distraction. The research half of Haskell will always push for research-oriented changes to GHC and base will follow in a piecemeal fashion.

Bodigrim commented 3 years ago

(Sorry for piecemeal answers, I'm doing too many things at once.)

Another point that concerns me is that the existing approach puts CLC members in a position of uncontrolled power.

That's not quite so, there are escape hatches in place. As outlined in PROPOSALS.md, base is owned by GHC developers (after all it's just a part of GHC source tree, right?), who for various reasons choose to outsource its evolution to CLC. If suddenly CLC goes mad, GHC developers have a supreme authority to strip CLC of its job or insist on other resolution.

Ericson2314 commented 3 years ago

@ocramz Yes.

Uou can do it today with enough stomach for CPP --- c.f. how glibc supports many different syscall ABIs but hopefully not that bad. I think it could well be worth it to start with that for anything that doesn't cleanly separate into a ghc-specific package for GHC to depend on. See what @kleidukos wrote in https://gitlab.haskell.org/ghc/ghc/-/issues/20647 for what such a split might look like.

Longer term, we would want to use backpack, and perhaps my idea in https://gitlab.haskell.org/ghc/ghc/-/wikis/Rehabilitating-Orphans-with-Order-theory, to make this less annoying.

hasufell commented 3 years ago

I propose:

Reasons for breaking changes to base

1. (security) bugfixes

No-brainers like the recent aeson breaking change. But may also include general bugs that require stronger types and thus breaking API.

The reason "security" is bracketed is because there's no definite definition of what a security bugfix is or is not. Any bug may potentially lead to an exploit, even if we're not aware of it. Thus I suggest not to focus too much on the word "security".

2. correctness improvements

This isn't about ergonomics or philosophical discussions like "is head a good idea?". It's about things that aren't mathematically/categorytheoretically correct or are fragile in the low-level sense: encoding issues, wrong representations (like FilePath), etc.

These may not be outright "bugs", because there's no obvious consensus about what's the right course, like with the broken ByteString IsString instance. But they may very well lead to bugs.

3. ergonomics improvements

This is the weakest category. Ergonomics can range from "let's replace head with safeHead" over things like the mentioned proposal or Monad of no return. These don't improve correctness per se and are not bugfixes, but make things more pleasant/stricter/haskell-ish for a large audience and not just core-library or base maintainers.

There must be a compelling reason why this can't be implemented outside of base and there should be a non-intrusive migration path and a 1-2 years migration period starting with compiler warnings.

This category requires a separate voting mechanism (see below).

Non-reasons for breaking changes

1. Performance improvements

If a performance improvement requires breaking API, then the improved functions/types should be implemented outside of base with a possible documentation adjustment in base, linking to those resources.

There may be exceptions to this rule, when fixing e.g. time complexity requires breaking API.

2. Trying out alternative API approaches (elegance)

There are many ways to express APIs. Some may be "better" than others, e.g. lazy IO is "evil" and I'd also like to see a proper streaming library replace those. However, those can be implemented outside of base and base shouldn't be a testbed for API experiments, but provide minimal building blocks.

3. ergonomics improvements that can be implemented outside of base

E.g. things that don't touch core instances/classes or core types. base shouldn't try to compete with alternative preludes.

Voting

I also suggest that 3. ergonomics improvements requires an unanimous CLC vote. It could be discussed whether ergonomics improvements require approval from core libraries maintainers and GHC maintainers as well, because this is the weakest category.

phadej commented 3 years ago

@hasufell: Do I understand your criteria right, that AMP,FTP,SMP which I guess all are Trying out alternative API approaches, and therefore won't be possible to do?

EDIT: I think that some voices are still not settled with these changes and are still frustrated.

I'd like to see changes of AMP scale to still be possible in the future (not easy, just possible).

hasufell commented 3 years ago

@phadej

AMP would probably fall under 2. correctness improvements, where I explicitly include math/CT correctness topics.

santiweight commented 3 years ago

I can only give half-baked thoughts currently (job search) - though the big guns of the Haskell are now here and my opinion will carry less (a good thing)!

I think something that is sorely missed in this discussion is an acknowledgment that the current severe uproar is really an indication that Haskell looks like a nightmare in production. I agree with many comments that "Eq of No Neq is not a big change". However, the overarching issue is the one that @chshersh raised: a prospective business faces the very serious and severe problem that they cannot predict whether their code will be easily-updatable to modern GHC versions. The issue with Eq of no Neq is not that the change itself causes churn or no churn - it's that no one can say how much churn maintainers and businesspeople will face in say the next year. It is seriously completely unpredictable, and subject to the whims of 6 people, even if those people are awesome! While there might be an answer, it is not in writing in a public location, and that simply won't cut it when millions of dollars are on the line.

I for one want to use Haskell at my day job, and while it is frustrating, the bay and many other cultures are safety-first, because product stability and predictable productivity is the name of the game. I have personally faced the "use Haskell at work" conversation, with an experienced Haskeller and self-exiled contributor who was pretty much terrified of using Haskell in production. The reality of the matter, whether you think it's reasonable or not, is that many Haskellers and non-Haskellers alike have no confidence that the decision making process is global or reasonable. While that is a statement of fear, economic recessions have occurred over similar amounts of uncertainty...

I personally would advocate for:

  1. A 2 year breaking changes window, and for breaking changes in popular APIs (stuff in Prelude, Control.Monad, etc.) to only occur every two years.
  2. For migrations to be semi-automated at the very least. I think we should keep non-migrators' work to pretty-much-only publishing a new package version on Hackage. (this is not black-white - more of a principle)
  3. Provide migration tools that are repeatable and can be hijacked for different proposals. Exploit ghc-exactprint and friends! (I have started work on this - let me know if you are interested in collaborating)
  4. Some kind of global strategy for base. I personally fear that the only reasonable strategy for base is one of extreme stability. Instead of proposing improvements to base, it is up to proponents of change, in my opinion, to make the work minimal for everyone else.
  5. Some alternative to CPP before we have any backwards-incompatible changes of any size.
santiweight commented 3 years ago

The barrier for changes needs to improve a lot more also. I believe a migration "strategy" won't cut it if we want people to have confidence in Haskell. The issue is not whether or not there is a strategy, but whether "I will have to spend my weekend updating my 50 libraries" which is quite an awful user experience.

If you advocate for a change to base, it should be up to you do the work by implementing the migration to the ecosystem, be that via PRs on Hackage or a migration tool. Defining that the migration is "easy" is still deferring the work to someone else. To be clear - I actually don't think Eq of no Neq is the issue, but the black box of future proposals which may or may not impact unsuspecting and time-constrained maintainers.

In other words - there is a very interesting technical PL problem here of how to avoid having downstream users pay the cost of trivial (non-bugfixing) migrations, such as Eq of no Neq or Monad of no return. I am personally starting to try to address this by finding ways to do PRs across all Hackage repos. That is only one sub-problem in a vast (but solvable) space and there are many moving pieces that Haskellers can work to address.

A good way, imo to think about this is thinking of base as a user-facing business. If any single one of my applications required me to spend even 2 hours migrating for some aesthetic change that came from an internal designer, I would immediately delete the application. Why then is it so surprising that Haskellers are scared at staying in Haskell... Many Haskellers feel that:

  1. Eq of no Neq will potentially affect them and their time whilst not gaining personally
  2. are scared that some future change will be far worse because no promises have been made

I love Haskell, and will stay because I love it, but if I were on the fence or outside of Haskell - I would delete the Haskell app extremely quickly if I felt that way!

phadej commented 3 years ago

FWIW, I don't think that aeson change was no-brainer. There was quite a lot of pressure to do something to get the twitter storm to calm down. And I suspect that then unhappy people are still unhappy. I'm also afraid that more base-process won't make now unhappy people any happier either. It will only make (other) people afraid to propose any changes (and new people to be on CLC).

ocramz commented 3 years ago

I'm also starting to think that, given the amount of uproar the Eq change proposal has caused, perhaps the CLC as an institution needs to be reformed.

Perhaps a broader voting mechanism needs to be put in place for breaking changes, like the yearly Haskell questionnaire that's advertised on most community channels.

Apparently assuming that all interested parties keep up to date with contentious issues on github is not enough.

Bodigrim commented 3 years ago

The barrier for changes needs to improve a lot more also.

@santiweight Let me reiterate what I said in https://github.com/haskell/core-libraries-committee/issues/12#issuecomment-967745158. Before removal of (/=) hits the ground, you are likely to update your packages twice already because of breaking changes in GHC or other boot libraries. Please advocate for less breakage on GHC side, and CLC could happily follow the suit.

If you advocate for a change to base, it should be up to you do the work by implementing the migration to the ecosystem, be that via PRs on Hackage or a migration tool.

It's not always possible in general, but for the particular case of (/=) CLC will provide practical help and guidance, it is just not happening overnight.

Perhaps a broader voting mechanism needs to be put in place for breaking changes, like the yearly Haskell questionnaire that's advertised on most community channels.

@ocramz Informal reactions under https://github.com/haskell/core-libraries-committee/issues/3#issue-1037180467 reveal that CLC acted in line with a popular vote. Understandably people unhappy with the outcome are more vocal at the moment, but the only cure is not to take any decisions at all.

The proposal was announced on libraries@ maillist in line with usual procedures, similar to many other proposals over the years, and gathered feedback there, on reddit and on github. In fact it was discussed far more widely and openly than it usually happens on maillists only.

ocramz commented 3 years ago

@Bodigrim , I know all of you are carrying out your mission to the letter and then some, and we should all be grateful for it.

I also know this is a recurring (and tiresome!) discussion but there seems to be a common, broader theme in those loud complaints that should be addressed. And to be clear, I'm 100% in favor of the proposal per se, but that's not the issue.

In the interest of reconciling the various strands of the community, I've started a thread on Discourse https://discourse.haskell.org/t/learning-from-the-no-in-eq-debate/3651

Bodigrim commented 3 years ago

@ocramz sorry, I do not really have capacity for one more thread, so let me comment here.

Yes, indeed, I understand the common theme of dissatisfaction with breaking changes in Haskell. I'm no academic, I am an industrial user myself, I maintain an almost 1 MLoC code base, I feel the pain. For years signalling about upcoming breakage in GHC, base and others was pretty much awful, exposing users to last minute migrations and broken build plans. People are afraid and tired of "we made a mess, now deal with it if you can".

I'm determined to improve this in areas of my responsibility. No changes without impact assessment. No changes without migration guide. No changes without patches for Stackage. No one will be left alone.

szabi commented 3 years ago
  1. A 2 year breaking changes window, and for breaking changes in popular APIs (stuff in Prelude, Control.Monad, etc.) to only occur every two years.

I think a 2 year breaking changes window sounds reasonable.

However, mandating braking changes only to happen every two years brings, in my experience, bigger trouble, as the effort to upgrade will be much higher (only once every two years, but much more breakage), and something that potentially cannot be budgeted in. There is a reason continuous delivery has become predominant and a best practice vs. the waterfall model across the industry.

santiweight commented 3 years ago

@Bodigrim Thank you for persisting on pushing for more consistency from GHC's side as well - it's a very good point that I agree with. When I have the time I will propose something similar in a GHC proposal, likely some time this week.

@szabi I think of the 2 year window as a good compromise until we can have better migration tools and the like, as has been mentioned by others in multiple places. In a dream world, this Eq of No Neq should be a very simple transformation of the ecosystem via cabal migrate eq-of-no-neq.

goldfirere commented 3 years ago

@santiweight Please do open a conversation about GHC breakage. I, for one, would love to learn more about how, precisely, GHC churn is breaking code -- and how these breakages are causing pain. Are they known breakages, as documented in the release notes? Are they accepted proposals? Are they unknown breakages? Are they on e.g. Stackage, or privately held code? Is it Template Haskell? From my perspective as a GHC developer, I see us carefully weighing breakage, using the proposals process, and helping people migrate. Yet it's clear that this is not working -- as evidenced by a number of people saying that GHC is causing more breakage here!

To be clear, I fully accept the premise that GHC is doing something wrong. My questions are not an attempt to discredit others or to find flaws in their reasoning. My questions are solely to understand better what's wrong so that we might put effort in the right place to fix it. Looking forward to the conversation.

hasufell commented 3 years ago

I, for one, would love to learn more about how, precisely, GHC churn is breaking code

I think the easiest way is to check head.hackage, which usually are 50+ patches per major GHC release.

Now imagine you're running a 1M LOC proprietary codebase with 5 developers. Keeping up with GHC releases will probably need to be planned and possibly disrupt the development. Something the HF could maybe ask some of their donors.

goldfirere commented 3 years ago

I just want to highlight @Bodigrim's little mantra from above (https://github.com/haskell/core-libraries-committee/issues/12#issuecomment-968366382):

No changes without impact assessment. No changes without migration guide. No changes without patches for Stackage. No one will be left alone.

That's lovely. Can it be worked into the README for the repo? Everyone should hear this message.

tonyday567 commented 3 years ago

There has been some pushback on social media about the Stackage/Hackage focus; that Hackage covers only a portion of affected prior work.

It is lovely, but others may hear it differently, as it excludes their care and attention.

Judging breaking changes is the issue of the day, but there are many benefits from just "deciding on core principles." Writing them down makes them contestable in advance of their usage.

As it now stands, the community is being a bit reactionary, and risks signalling that squeaky wheels get the good oil, that decisions and judgement can be varied, and resources acquired by simply (and loudly) espousing a preferred set of principles.

simonmar commented 3 years ago

I, for one, would love to learn more about how, precisely, GHC churn is breaking code -- and how these breakages are causing pain. Are they known breakages, as documented in the release notes? Are they accepted proposals? Are they unknown breakages? Are they on e.g. Stackage, or privately held code? Is it Template Haskell? From my perspective as a GHC developer, I see us carefully weighing breakage, using the proposals process, and helping people migrate. Yet it's clear that this is not working -- as evidenced by a number of people saying that GHC is causing more breakage here!

It takes us about 1 person-year to upgrade GHC at Meta. The last upgrade we did (8.4 to 8.8) we kept track of how much time was being spent on which parts of the process. The breakdown was roughly like this (we kept track of more detail but I don't want to try to paraphrase all of it here):

Yes we are a special case in that we modify GHC. But that only accounts for maybe a third of the time spent doing the upgrade.

Some observations:

I don't really expect this to influence how we do things in GHC or the wider Haskell community, other than to try to foster a bit of sympathy for those of us in industry who have massive gobs of code and dependencies on everything, and are trying not to fall behind. We need to be able to change things, I'm on board with that. Perhaps changing things at a slower cadence would be helpful, though.

Let me offer at least a bit of perspective on what doesn't help:

nomeata commented 3 years ago

In fact from our perspective it would be better not to have the warning stage, it's just extra churn. We'll fix it when it breaks.

That is an interesting relevation, and I wonder how widespread that is. I expect that well tended code is likely kept warning-free, so to their maintainers, the introduction of the warning is already the “now do something” stage. And for code that is just kept alive, warnings are likely ignore until one cannot. So I wonder, how useful are they after all? Are they “just” a more aggressive form of release notes then?

szabi commented 3 years ago

I was equally surprised, but we cannot discount the experience of an industrial expert. (But must be aware that this is only one opinion of many).

My thoughts about deprecation warnings are:

In this context, I think "warnings" can be a kind of a dependable (if imperfect) tracking tool for (a subset of) technical debt.

As you said, "well tended code", where the maintainers have actually a budget and management a commitment for maintaining will have a policy for keeping code warnings-free: therefore deprecation warnings are resolved immediately. However, given other constrains, project management can decide that this is not a priority, and accrue technological debt: warnings can be ignored -- for now -- but need to be addressed later.

In this sense, the "warning" stage affords flexibility, while exposing the truth, preventing a complete blind-flight, preventing the issue being swept under the rug, and making it harder to ignore the fact. Would one forgo the warning stage and only throw an error at removal, the product managers would be in a kind of "blind flight". Warnings give us -- theoretically -- a tool for debt assesment and management.

In practice I have seen too that warnings are more often than not ignored "due to budgetary reasons", and this seems to be the experience of @simonmar as well. Nevertheless, I think removing the warning stage is a net loss: we loose information we could -- and indeed can -- utilize: I have certainly seen management be convinced to reduce Mount Warning to eliminate some debt. This convincing was possible due to warnings being raised: after all, "# of warnings" is a very concrete (if imperfect) measure. And we know, management likes everything that's measurable, and won't be easily convinced by "your hunch"/"experience" alone.

Forgoing the warning stage and only raising errors when the feature is removed / breaking change introduced (paraphrasing Simon "because we don't read release notes anyway and we'll fix it when it's actually broken) would be completely equivalent to not having a phasout plan and just introducing breakage ad-hoc without any notice.

Given how well received the stricter deprecation policy was received in the wider Haskell community, and given how typical enterprise project languages are praised not only for their backward compatibility, but planable and predictable deprecation policies, I just can't believe that dropping back to what is effectively immediate and ad-hoc breakage (not raising warnings, just going for error would be effectively this, notage of deprecation in future removal in release notes don't seem to matter as "noone reads the release notes anyway") is indeed desirable.

ocramz commented 3 years ago

Very interesting to read @simonmar 's perspective -- it would be also enlightening to hear from smaller entities than Facebook that perhaps cannot afford to throw whole person-years at migrations.

Startups : What are your coping/evolution strategies with breaking changes in dependencies ? Or do you avoid updating altogether?

tomjaguarpaw commented 3 years ago

For the purposes of looking at Simon's comment with "scientific control" I think it would also be beneficial to gather evidence of the cost of upgrading in other ecosystems. How many person years are required to upgrade Java, Go, C++, Python, for example?

simonmar commented 3 years ago

Upgrading the C++ toolchain probably takes a lot more effort overall, but there's a lot more C++ code of course. I don't have any actual data I can share unfortunately.

I realised I should explain a little more why release notes and warnings don't end up being as useful as you might think. The larger the codebase, the more you want to centralise the job of maintaining and upgrading the toolchain and libraries. So it'll be one person or a small number of people doing most of the work.

Not only are there a lot of release notes and announcements to read during an upgrade, but since the person doing the upgrade isn't the person who wrote the application code, they are likely to miss important things or not realise the implications of changes mentioned in the release notes anyway. Therefore instead of trying to proactively fix things, we rely heavily on automation: turn on -Wall -Werror and use HLint extensively, fix compile errors and hope that batteries of tests and benchmarks will catch anything else that gets through. Project owners would normally get a chance to check that an upgrade is OK, but typically they'll just look at the test/benchmark results. If your tests and benchmarks aren't catching problems, then they're not good enough!

The changes that worry me the most are things like this: https://www.haskell.org/ghc/blog/20210607-the-keepAlive-story.html

The compiler, warnings, and HLint are not going to catch that. If we're lucky the benchmarks will catch the performance regression before it gets into production, but even if the benchmarks catch it, tracking down the cause of the regression will be an adventure for somebody. The only way around that is for someone to be reading the release notes carefully and proactively acting on it - fortunately I know about this one so when we do the upgrade to 9.0+ I'm going to have to go around and change all the withForeignPtrs to unsafeWithForeignPtrs. Here's hoping I remember :)

davean commented 3 years ago

Yes it's the semantic changes that scare me - and I do read the release notes. These are the ones that need careful consideration and planning. They're also often the ones - like the noted one - we can't really avoid.

We should focus our planning and solutions more on those.

On Fri, Nov 19, 2021 at 12:14 PM Simon Marlow @.***> wrote:

Upgrading the C++ toolchain probably takes a lot more effort overall, but there's a lot more C++ code of course. I don't have any actual data I can share unfortunately.

I realised I should explain a little more why release notes and warnings don't end up being as useful as you might think. The larger the codebase, the more you want to centralise the job of maintaining and upgrading the toolchain and libraries. So it'll be one person or a small number of people doing most of the work.

Not only are there a lot of release notes and announcements to read during an upgrade, but since the person doing the upgrade isn't the person who wrote the application code, they are likely to miss important things or not realise the implications of changes mentioned in the release notes anyway. Therefore instead of trying to proactively fix things, we rely heavily on automation: turn on -Wall -Werror and use HLint extensively, fix compile errors and hope that batteries of tests and benchmarks will catch anything else that gets through. Project owners would normally get a chance to check that an upgrade is OK, but typically they'll just look at the test/benchmark results. If your tests and benchmarks aren't catching problems, then they're not good enough!

The changes that worry me the most are things like this: https://www.haskell.org/ghc/blog/20210607-the-keepAlive-story.html

The compiler, warnings, and HLint are not going to catch that. If we're lucky the benchmarks will catch the performance regression before it gets into production, but even if the benchmarks catch it, tracking down the cause of the regression will be an adventure for somebody. The only way around that is for someone to be reading the release notes carefully and proactively acting on it - fortunately I know about this one so when we do the upgrade to 9.0+ I'm going to have to go around and change all the withForeignPtrs to unsafeWithForeignPtrs. Here's hoping I remember :)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/haskell/core-libraries-committee/issues/12#issuecomment-974255170, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABZLSYUAWLBFLYFT23YY53UM2AYFANCNFSM5HCTHEQQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Ericson2314 commented 3 years ago

I would recommend @simonmar's team and others take advantage of things like -Wcompat by toggling whether they fail builds. I agree with the basic "fail fast" principle that non-errors get ignored, but by doing things like -Werror=compat and -Wno-error=compat one can still be "failure-driven" and yet not deal with every sort of breakage at once.

At Obsidian, we were in a bit a special position in that GHCJS is usually the limiting factor preventing upgrades, blocking us after the ecosystem has by-and-large resolved most regular platform-agnostic issues. But we try to take an analogous separating our Nixpkgs and GHC upgrades so we are never upgrading both at the same time. We sometimes get stuck on weirder errors than intended breakage, and this "tick tock" alternation of what we are upgrading also helps troubleshooting, and the "upgrade experiment" is better controlled.

In particular, the worst part about upgrades is not to the total time investment, but that the work is so nonparallel: when fixing failures, once typically only gets a few failures at a time. With -Wno-error=compat then -Werror=compat, a vanguard group of people can go through getting things to merely build, more permissively, and a second group can follow fixing the error warnings. Beyond the parallelism and critical path length benefits, lIke "scouting" in real time strategy games, this lifts the "fog of war" of the upgrade projects, revealing any major impediments sooner.

Similarly, we should:

  1. Hopefully release GHC more often, in which case there are similarly more and smaller "waves of upgrades" propagating through, and thus better incrementality, parallelism and troubleshooting.
  2. Decouple base and GHC as I have written in https://discourse.haskell.org/t/pre-pre-hftp-decoupling-base-and-ghc/3727/8, so those upgrades can also be separated for the same reasons.

Here's the most important thing to remember: the closer large organizations are to the latest release, the better incentivized they are to help out with upstream FOSS development, as any contributions they make will trickle back to them and benefit them sooner. We want to get that upgrade latency to an absolute minimum, independent of the throughput of breaking changes.

jberryman commented 3 years ago

I've worked at three Haskell startups. I more or less agree with Simon's overall sentiment, and the observation that warnings don't save much time (the exception being when they can contain more information than an error would); certainly we won't notice warnings in dependencies.

But the particulars sound very different from my experience at much smaller companies, even though e.g. we've been running a fork of ghc with patches.

At hasura we have 400-500 modules primarily in a single mono repo. We're in the process of upgrading to 9.2 from 8.10; upgrading our own code is probably a day's work, while fixing dependencies and (in particular) getting those changes upstreamed, temporarily vendoring and tracking PRs, etc... is a soul-crushing and thankless (like, being actually scolded) slog. When we're done we'll need to have a discussion about whether our upgrade will be blocked on hls, and then maybe try to help there (9.0 is still not quite fully supported iirc)

I think in general the breaking changes to ghc and core libs are really thoughtful, well-considered, and reasonable.

But I think the principle I'd propose is: if you break it, fix hackage. If that's too difficult to do by hand, then write a script (if that's too difficult write a tool). If maintainers don't want to merge a change to support an unreleased ghc version, figure out how to automate staging PRs from head.hackage maybe.

I don't think that's too much to aspire to (at least for "small" breaking changes, and for the popular/active subset of hackage)

EDIT: actually one benefit of changes that come with a warning the version prior is that it does allow you to open a PR ahead of time without maintainers yelling at you. But again we're only going to start fixing other people's libraries when we go to upgrade and find they break

treeowl commented 3 years ago

@simonmar

Warning of upcoming changes in release notes or announcements is good, but it doesn't save us any time. We're probably not going to read the release notes for all the tools and packages we're upgrading until we're actually doing the upgrade, and maybe not even then - we'll find out when we hit the error.

That's ... a bit troubling. If you don't read release notes until you notice something "doesn't work", that means you could miss important but subtle changes in behavior. Ouch!

vdukhovni commented 3 years ago

In fact from our perspective it would be better not to have the warning stage, it's just extra churn. We'll fix it when it breaks.

That is an interesting relevation, and I wonder how widespread that is. I expect that well tended code is likely kept warning-free, so to their maintainers, the introduction of the warning is already the “now do something” stage. And for code that is just kept alive, warnings are likely ignore until one cannot. So I wonder, how useful are they after all? Are they “just” a more aggressive form of release notes then?

Personally I like to address future issues on an ongoing basis, as they come up, rather than big-bang. So warnings are helpful. And build and maintain compatibility with future releases to which I'm not necessarily immediately ready to switch. Approaches vary...