search

haskell / base16-bytestring

Fast base16 (hexadecimal) encoding and decoding for Haskell bytestrings.
Other
27 stars 10 forks source link

Broken reverse dependencies since 1.0.0 #14

Closed hasufell closed 4 years ago

hasufell commented 4 years ago

I believe the release broke at least:

and as a consequence all deps that depend on text-conversions:

and all deps that depend on string-interpolate:

Yes, I believe checking reverse dependencies is what we should do before a release: https://packdeps.haskellers.com/reverse/base16-bytestring

This will just be a revision update, so a hackage trustee could unbreak this. @phadej ?

phadej commented 4 years ago

@hasufell, I have stepped down from being a Hackage Trustee.

hasufell commented 4 years ago

@gwils

gwils commented 4 years ago

I also stepped down from trusteeship. @emilypi

emilypi commented 4 years ago

I'm not a trustee, but I'm happy to help @hasufell

emilypi commented 4 years ago

Yes, I believe checking reverse dependencies is what we should do before a release: https://packdeps.haskellers.com/reverse/base16-bytestring

FWIW, i had the issue out for months, and I made two announcements publicly 3 months ago that i was going to make radical changes to the library. See: here.

Short of personally messaging each maintainer of a downstream dependency of this library, which is an unreasonable request to make of any maintainer, I did what I needed to do. This includes offering my services:

If you want to help with the next changeset, please feel free to talk to me on irc (i am topos) or email me or get involved with the v1.0 omnibus PR here: https://github.com/haskell/base16-bytestring/pull/10

It was everyone else who failed to pay attention.

mightybyte commented 4 years ago

@hasufell None of the packages you mention have upper bounds on base16-bytestring. If they had appropriate PVP-compliant bounds, they would not have been broken by this release. This is not a problem with base16-bytestring. This is a problem with the downstream packages that failed to put appropriate version bounds.

I'm a huge advocate of backwards compatibility but from time to time breaking changes do make sense. It is the responsibility of downstream packages to use dependency bounds so that they will be robust in these situations.

hasufell commented 4 years ago

This is not a problem with base16-bytestring. This is a problem with the downstream packages that failed to put appropriate version bounds.

We could have a long discussion about this, but either way... it isn't enforced and the end user doesn't really care who is to blame in my opinion. So I personally think checking reverse dependencies is something we should do, if it's reasonable.

mightybyte commented 4 years ago

@hasufell It's not enforced, so guess what...buyer beware. And in this case the buyer is you. If you depend on packages that don't put upper bounds, you're going to run into this eventually.

It's absolutely not reasonable to expect maintainers to check reverse dependencies. This package has 160 reverse dependencies. There's no way they can check them all. Remember, they're doing this work for free. What is reasonable is preferring maintainers who don't make breaking changes very often. And there's no indication that this is going to become a trend in this case.

hasufell commented 4 years ago

Well, I disagree with you on that. I'd have put up a build for those reverse deps on my PC and let it run over night. 160 deps isn't much in haskell and writing a script for that should be well feasible. That's also my experience as a linux distro maintainer.

I'm not putting blame on anyone, but explaining that CI can help with this as well. There are many ways to get PVP wrong and accidentially break reverse deps, even if there are upper bounds.

mightybyte commented 4 years ago

I don't care whether it's theoretically possible to do it in this case. What you are describing simply doesn't scale. There is no limit to how many people / packages can depend on your package. It's patently absurd to suggest that upstream is responsible for the decisions of everyone downstream. You know what is scalable? Owning the implications of the dependencies that you choose to add to your project and the risks associated with them.

emilypi commented 4 years ago

Feel free to donate to my github sponsorship program with the amount needed to afford another box to run things. I am on an underpowered work laptop 99% of the time. The other 1% is spent ssh'ing into a $5 droplet (from said laptop). Neither can support routine CI builds of any magnitude (which is why I outsource to Travis!). I do not own a computer otherwise.

hasufell commented 4 years ago

I don't care whether it's theoretically possible to do it in this case.

I'm aware that the haskell ecosystem is particularly eager in breaking API and offloading things to PVP. But I believe that didn't scale either: it's the reason we got cabal hell, stackage, nix-style builds, hackage revisions and other workarounds and coping tools. I don't think this is mainly an issue with the PVP spec, but the eagerness of breaking API in general and not using deprecation phases and maybe that one historical split in the community.

What you are describing simply doesn't scale.

Whether you believe in PVP or not... testing reverse deps is a good thing if you have the capacity. And we can talk somewhere else about scaling issues, because I believe these aren't hard issues. Again: Linux distros do this since a few decades. Stackage does this too, in small scope. We can define the scope for hackage too.

Also, it's true there are other ways to deal with this... and these go deeper into the tooling foxhole: https://dependabot.com/rust/ but I'd consider this complementary. A certain set of core packages on hackage should have their reverse deps built prior to releases.

mightybyte commented 4 years ago

@hasufell I look forward to your contributions to improve the state of the CI for this package.

cartazio commented 4 years ago

has ANYONE pinged / opened a trustees ticket to track fixing rev deps? I no longer have the perms to do so, else i'd happily do the ecosystem surgery plus downstream notifications.

agreed that testing downstream clients that are publicly available is idea, but definitely resource intensive and requires tooling and actually support

emilypi commented 4 years ago

@cartazio yes: https://github.com/haskell-infra/hackage-trustees/issues/280

@hasufell if you want to set up community revdep CI for all major haskell packages I would enjoy using it! Or even for this repository. Feel free to take out an issue to spec it out. I'll approve, and then we'll sort out funding for it. If you'd like to donate a machine to it, that would be very helpful. In fact, why don't we meet up this week to discuss it? I'm available every day, all days, as long as it doesn't conflict with my meetings.

hasufell commented 4 years ago

@emilypi I have a few ideas. The first would be to write a script that fetches reverse deps and sets up a dummy project with all those as local packages and then forces the development version of the main library via cabal.project (with allow-newer). The next step would be to compare failures to previous failures to see if there are new failures and actual regressions (regressions are only within package bounds). This could first run on travis if the build time is less than 2 hours.

But given the partly negative reactions here, I'm not sure people are interested in that approach.

mightybyte commented 4 years ago

I'm not sure people are interested in that approach.

I am not a maintainer of this package, so my comments have nothing to do with what the maintainers will accept. @emilypi just said "I would enjoy using it", and "that would be very helpful". That sounds like pretty clear openness to your suggestions to me.

emilypi commented 4 years ago

I think the utility of this conversation has run its course.

@hasufell feel free to get in touch with me at my email to actually build the thing, and we can start off on the right foot in a more appropriate setting.