search

Homebrew / homebrew-core

🍻 Default formulae for the missing package manager for macOS (or Linux)
https://brew.sh
BSD 2-Clause "Simplified" License
13.77k stars 12.45k forks source link

They Shoot Horses, Don't They? #66360

Closed fxcoudert closed 3 years ago

fxcoudert commented 3 years ago

The problem: old unbuildable formulas

(Edit: the list was updated 2020-12-20)

325 winexe
244 connect
125 vera++
102 zssh
101 cabocha
73 shntool
73 pcre++
73 hardlink-osx
62 git-hooks
51 mpack
41 makeicns
15 git-cal
0 mdxmini

They need to be fixed, or removed. Some of them are already heavily patched. Because they haven't been updated in 4 years, they're unlikely to be actively maintained.

Suggested solution

We have limited volunteer availability, so I would suggest:

What do you think @Homebrew/core ?

claui commented 3 years ago
  • All those that have less than 20 monthly installs are removed

Fixed that for you.

  • If someone wants to add them back, they are welcome! They will be treated as new formulas

Sounds reasonable.

carlocab commented 3 years ago

I agree with removing them all, though I don't mind the initial suggestion of keeping some of them too.

If someone wants to add them back, they are welcome! They will be treated as new formulas

Can we document this in the PR where they are removed? (Or I suppose a link to this issue in that PR would suffice.) It seems the current message someone who tries to install a removed formula would see is something like

==> Searching for a previously deleted formula (in the last month)...
pgplot was deleted from homebrew/core in commit 246cfef514:
  pgplot: delete due to license issues (https://github.com/Homebrew/homebrew-core/issues/64332)

To show the formula before removal run:
  git -C "$(brew --repo homebrew/core)" show 246cfef514^:Formula/pgplot.rb

If you still use this formula consider creating your own tap:
  https://docs.brew.sh/How-to-Create-and-Maintain-a-Tap
mitchblank commented 3 years ago

A couple of those are on the list of bottling issues I've been looking at:

A couple others I was planning to take a quick look at tonight (hfsutils, ipv6toolkit)

mitchblank commented 3 years ago

hfsutils and ipv6toolkit were just broken due to stale download URLs; both build OK on Big Sur once that is fixed (#66371, #66372 respectively)

MikeMcQuaid commented 3 years ago
  • All those that have less than 20 monthly installs are removed

s/removed/deprecated or disabled

  • If someone wants to add them back, they are welcome! They will be treated as new formulas

👍🏻

fxcoudert commented 3 years ago

deprecated or disabled

If a formula is not building, not maintained, what's the point of disabled? To keep it in our history? That's why we use git. (Otherwise these disabled formulas accumulate for no reason)

MikeMcQuaid commented 3 years ago

If a formula is not building, not maintained, what's the point of disabled? To keep it in our history? That's why we use git. (Otherwise these disabled formulas accumulate for no reason)

We can remove disabled formulae once they've been disabled for a while. They are a significantly nicer experience for end-users to extract/modify/fix than formulae that are removed.

SMillerDev commented 3 years ago

Disabled will tell users it's not supported while removed is a surprise on the next attempted install.

fxcoudert commented 3 years ago

Can we set a limit on how long we will keep things disabled before they are removed? 1 year?

scpeters commented 3 years ago

Can we set a limit on how long we will keep things disabled before they are removed? 1 year?

yeah, you can add a date to deprecate! and disable! calls. Before the deprecate! date, no errors are given, and afterwards, deprecation warnings are given when installing a formula. I believe that adding a disable! date will deprecate the formula before the date and disable it afterwards. I think it's reasonable to add disable! statements with the date of Big Sur's release and then remove them outright after a fixed period of time. I would hope that disabled formulae are just ignored by CI, so that the cost of not deleting them is low.

SMillerDev commented 3 years ago

I would hope that disabled formulae are just ignored by CI, so that the cost of not deleting them is low.

They are ignored, I'm fine with keeping disabled formula for a year. We just haven't had the feature long enough to get to that yet.

fxcoudert commented 3 years ago

Could we also agree that disable! and deprecate! should systematically have a date, so we can start the clock and not keep them forever?

SMillerDev commented 3 years ago

I agree they should have a date. And I think I should be able to write an audit that checks it they should be removed.

fxcoudert commented 3 years ago

An audit to mandate dates should be good. Checking for expiration of dates seems likely to show up in unrelated PRs, causing trouble, so probably not wanted. Also, I can image there would be cases where we have good reasons to remove earlier or later?

MikeMcQuaid commented 3 years ago

Can we set a limit on how long we will keep things disabled before they are removed? 1 year?

Makes sense to me. I don't have strong feelings on what this date is.

Could we also agree that disable! and deprecate! should systematically have a date, so we can start the clock and not keep them forever?

The date is when they kick in, not when they are removed.

An audit to mandate dates should be good. Checking for expiration of dates seems likely to show up in unrelated PRs, causing trouble, so probably not wanted. Also, I can image there would be cases where we have good reasons to remove earlier or later?

The best fix for me (albeit a slightly more involved one) is having a GitHub Actions job which deletes these formulae (or opens a PR to do so) when they hit the relevant date.

fxcoudert commented 3 years ago

The date is when they kick in

Yes, but having a date systematically (which is not always the case currently), even if it's today or in the past, means we know how long they've been broken/disabled.

MikeMcQuaid commented 3 years ago

Yes, but having a date systematically (which is not always the case currently), even if it's today or in the past, means we know how long they've been broken/disabled.

So if we want to deprecate/disable immediately you'd see setting this to e.g. today's date? This would make sense to me. If so, I'd suggest we consider deprecating the non-date format.

SMillerDev commented 3 years ago

If so, I'd suggest we consider deprecating the non-date format.

I think that's a good idea.

fxcoudert commented 3 years ago

So if we want to deprecate/disable immediately you'd see setting this to e.g. today's date?

This is what I've been doing, yes.

MikeMcQuaid commented 3 years ago

I think that's a good idea.

Cool, done (but can't merge until next minor release): https://github.com/Homebrew/brew/pull/9478

fxcoudert commented 3 years ago

Dates being added to all formulas where they were missing: https://github.com/Homebrew/homebrew-core/pull/66549

fxcoudert commented 3 years ago

Update: remaining problematic formulas

cabocha cdrtools connect git-cal hardlink-osx mdxmini mpack pcre++ shntool vcprompt vera++ winexe zssh
mitchblank commented 3 years ago

remaining problematic formulas

The ones on that list I've previously looked at:

fxcoudert commented 3 years ago

cdrtools: their "latest version" link downloads cdrtools-3.02a09, so maybe we could consider that stable (it was released in 2017)?

fxcoudert commented 3 years ago

pcre++ fails to download on CI, but works for me:

==> FAILED
==> Downloading https://www.daemon.de/idisk/Apps/pcre++/pcre++-0.9.5.tar.gz
curl: (7) Failed to connect to www.daemon.de port 443: Operation timed out
EricFromCanada commented 3 years ago

The author of cdrtools releases snapshots of it and several others as part of the schilytools package, so it appears that what's listed under the cdrtools "alpha" folder is what's presented as a "release".

mitchblank commented 3 years ago

cdrtools-3.02a09, so maybe we could consider that stable (it was released in 2017)?

Just tried it... still getting:

        ==> COMPILING "cdda2wav.o"
In file included from cdda2wav.c:102:
./mytype.h:29:1: error: unknown type name 'error'
error need an integer type with 32 bits, but do not know one!
^

This is very often autoconf related and it definitely has xcode12-related breakages there:

$ grep -c -e '-Werror,-Wimplicit-function-declaration' $( find . -name config.log )
./incs/i386-darwin-clang/config.log:159
./cdda2wav/OBJ/i386-darwin-clang/config.log:0

I'll try seeing if some $CFLAGS massage helps...

mitchblank commented 3 years ago

OK, opened #66661 for cdrtools. Builds/tests for me but we'll see what CI thinks of it

mitchblank commented 3 years ago

Status of a few others from the "updated" list:

fxcoudert commented 3 years ago

@mitchblank I've updated the list. Re hardlink-osx I'm 👍 for disabling it, especially as the thread you linked raises questions of data loss!

mitchblank commented 3 years ago

I've updated the list.

You still have dupseek on the list, but we've had a Big Sur bottle for that as of 12 days ago: #66509

mitchblank commented 3 years ago

Took a quick look at mpack; it just needed a new download UR: #67290

chenrui333 commented 3 years ago

looks like I completely missed this discussion, do we have some guidelines of how to use deprecate! vs disable!? One tricky thing is the repo got removed (I remember there are some utilities suffered from that issue).

bayandin commented 3 years ago

My general understanding is:

SMillerDev commented 3 years ago

In my idea it's deprecate! for software that SHOULD no longer be used, upstream depreciation or archival is the clearest example.

And disable! for anything that either MUST not be used anymore due to license issues or prolonged depreciation, or CAN'T be used anymore due to build issues.

Rylan12 commented 3 years ago

I opened a PR with some documentation about this in https://github.com/Homebrew/brew/pull/10144. Feedback welcome!

fxcoudert commented 3 years ago

I'm closing this issue as we've made great progress on this list, thanks everyone