search

Homebrew / homebrew-core

🍻 Default formulae for the missing package manager for macOS (or Linux)
https://brew.sh
BSD 2-Clause "Simplified" License
13.75k stars 12.44k forks source link

Remove versioned dependency conflicts #65831

Closed Rylan12 closed 2 years ago

Rylan12 commented 3 years ago

We have an audit to ensure that a formula doesn't depend on multiple versions of the same formula (for example, a formula cannot depend on both python@3.8 and python@3.9). Unfortunately, there are currently 6 formulae that have conflicting versioned dependencies. We should try to remove these if we can.

The problematic formulae are:

This issue was prompted by https://github.com/Homebrew/brew/pull/9314 and https://github.com/Homebrew/homebrew-core/pull/65712

carlocab commented 3 years ago

fdroidserver depends on python@3.8 through sc3md. 

❯ brew deps --tree fdroidserver
fdroidserver
├── freetype
│
│   [snip]
│
├── s3cmd
│   └── python@3.8

sc3md is currently waiting for a version bump so its dependency can be upgraded to python@3.9:

https://github.com/Homebrew/homebrew-core/blob/1fed3baac24b4457b10eaeab2c1532c47dda2dea/Formula/s3cmd.rb#L24-L29

Backporting patches may be possible for someone familiar with s3cmd.

carlocab commented 3 years ago

I had a look at the openjdk conflicts. It seems fixing some of them would introduce other ones. Perhaps this is already a minimal set of versioned conflicts in the dependency tree?

Here's sqoop's dependency tree:

❯ brew deps --tree sqoop
sqoop
├── coreutils
├── hadoop
│   └── openjdk
├── hbase
│   ├── lzo
│   └── openjdk@11
├── hive
│   ├── hadoop
│   │   └── openjdk
│   └── openjdk
├── openjdk
└── zookeeper
    ├── openjdk
    └── openssl@1.1

Switching hbase to openjdk does not seem possible: https://hbase.apache.org/book.html#java

However, since openjdk@11 is the LTS release, it seems likely that hadoop, hive, and zookeeper can all be switched to openjdk@11. (The hadoop docs actually suggest using openjdk@8 or openjdk@11, preferably the former.)

Doing this will affect at least these formulae:

❯ parallel brew deps --tree ::: flume mahout
mahout
├── hadoop
│   └── openjdk
└── openjdk

flume
├── hadoop
│   └── openjdk
└── openjdk

Switching hadoop to openjdk@11 will require migrating mahout and flume as well. Or, said differently, fixing the conflict for sqoop introduces at least two new versioned conflicts.

Here's predictionio's dependency tree:

❯ brew deps --tree predictionio
predictionio
├── apache-spark
│   └── openjdk@11
├── elasticsearch@6
│   └── openjdk@8
│       └── freetype
│           └── libpng
├── hadoop
│   └── openjdk
├── hbase
│   ├── lzo
│   └── openjdk@11
└── openjdk@8
    └── freetype
        └── libpng

Assuming we wanted to migrate sqoop's tree to openjdk@11, we would have to do the same for elasticsearch@6 and predictionio. The latter seems possible; I can't find docs for the former.

To be clear, I'm happy to help with fixing the conflict if it can be fixed, but it seems like a decision needs to be taken for a whole dependency (sub)tree as to whether they're all migrated to openjdk@11 or openjdk@8, so I'd want to know which one's a better idea first before working on a PR.

Rylan12 commented 3 years ago

Yeah, these may not be immediately actionable. I only opened this issue because the TODO: line was removed in another PR and we didn't want to lose the reminder to look at it.

Perhaps this is already a minimal set of versioned conflicts in the dependency tree?

That's definitely possible. The goal, obviously, is 0 but...

Thanks for looking into this, though. I don't think it makes sense to modify the dependencies of all these formulae just to satisfy the one audit for this one. I think we'd rather have as much stuff working on the latest version as possible, so if removing the conflict involves a lot of reworking dependencies and downgrading when maybe not necessary (audit aside), I don't think it makes sense to do it.

carlocab commented 3 years ago

Sure. Let me know if there's anything more I can do to help.

gromgit commented 3 years ago

anjuta pulls in guile@2 via autogen, whose current configure only checks for Guile versions <= 2.2. (This was in fact reported early this year, and was probably the genesis of guile@2 itself: #49121)

A corresponding bug was filed with autogen's own bug tracker, and the latest comment on Sep 14 2020 is instructive:

With the next release, AutoGen will no longer check the Guile version. It will now be simply expected that Guile will be at or after version 2.2 and that they won't change the interface anymore.

So I guess it'll be fixed in the fullness of time. :grin:

gromgit commented 3 years ago

As for visp, it pulls in vtk@8.2 via pcl, and there's an in-progress upstream PR to rev that: https://github.com/PointCloudLibrary/pcl/pull/4262

gromgit commented 3 years ago

Finally, agda is a temperamental beast, judging from recent PRs for it or ghc, so until it actually passes an overdue version bump, it's probably best not to even look at it funny. :grin:

MikeMcQuaid commented 3 years ago

To be clear, I'm happy to help with fixing the conflict if it can be fixed, but it seems like a decision needs to be taken for a whole dependency (sub)tree as to whether they're all migrated to openjdk@11 or openjdk@8, so I'd want to know which one's a better idea first before working on a PR.

@carlocab If in doubt: pick the newer one.

Hjsurk commented 3 years ago

Hello,

Just wondering if this problem is the same issue as the one reported above. When I run the following command:

brew install kabel/php-ext/php@7.4-imap

I will get a message saying php-imap for PHP 8 has already been installed.

Warning: kabel/php-ext/php-imap 8.0.0 is already installed and up-to-date
To reinstall 8.0.0, run `brew reinstall php-imap`
carlocab commented 3 years ago

@Hjsurk hard to say, but I don't think so (I can't find a versioned dependency conflict in kabel/php-ext/php@7.4-imap's dependency tree. It's probably best to check at https://github.com/kabel/homebrew-php-ext.

vnckppl commented 3 years ago

We have an audit to ensure that a formula doesn't depend on multiple versions of the same formula (for example, a formula cannot depend on both python@3.8 and python@3.9). Unfortunately, there are currently 6 formulae that have conflicting versioned dependencies. We should try to remove these if we can.

The problematic formulae are:

  • [ ] agda

    • Conflicts: ghc, ghc@8.8

  • [ ] anjuta:

    • Conflicts: guile@2, guile

  • [ ] fdroidserver:

    • Conflicts: python@3.9, python@3.8

  • [ ] predictionio:

    • Conflicts: openjdk@11, openjdk@8, openjdk

  • [ ] sqoop:

    • Conflicts: openjdk, openjdk@11

  • [ ] visp:

    • Conflicts: vtk, vtk@8.2

This issue was prompted by Homebrew/brew#9314 and #65712

Is this an example of the guile conflict you mention:

brew install mu
==> Downloading https://github.com/djcb/mu/archive/1.4.13.tar.gz
Already downloaded: /home/vincent/.cache/Homebrew/downloads/d87cf2f510c663de6b4a7dbf34ef00c4607baa664591111607bbcf41c7150987--mu-1.4.13.tar.gz
==> autoreconf -ivf
==> ./configure --prefix=/home/linuxbrew/.linuxbrew/Cellar/mu/1.4.13 --with-lispdir=/home/linuxbrew/.linuxbrew/Cellar/mu/1.4.13/share/emacs/site-lisp/m
Last 15 lines from /home/vincent/.cache/Homebrew/Logs/mu/02.configure:
checking if the linker (/home/linuxbrew/.linuxbrew/Homebrew/Library/Homebrew/shims/linux/super/ld -m elf_x86_64) is GNU ld... yes
checking for shared library run path origin... done
checking 32-bit host C ABI... no
checking for the common suffixes of directories in the library search path... lib,lib
checking for guile-2.2... yes
checking for pkg-config... (cached) /home/linuxbrew/.linuxbrew/opt/pkg-config/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
configure: checking for guile 2.2
configure: found guile 2.2
checking for guile-2.2... no
checking for guile2.2... no
checking for guile-2... no
checking for guile2... no
checking for guile... /home/linuxbrew/.linuxbrew/opt/guile/bin/guile
configure: error: found development files for Guile 2.2, but /home/linuxbrew/.linuxbrew/opt/guile/bin/guile has effective version 3.0

READ THIS: https://docs.brew.sh/Troubleshooting
gromgit commented 3 years ago

Yes, see https://github.com/Homebrew/homebrew-core/issues/65831#issuecomment-735316686

$ brew deps mu
[...]
autogen
[...]

brew unlink guile@2, then try brew install mu again. Never mind, guile@2 is already keg-only, so that's basically a no-op.

carlocab commented 3 years ago

This should fix agda, I think. https://github.com/Homebrew/homebrew-core/pull/67014

Update: Fixed! Funny that this is the first one that's crossed off your list, given

Finally, agda is a temperamental beast, judging from recent PRs for it or ghc, so until it actually passes an overdue version bump, it's probably best not to even look at it funny. 😁

(But I'll have to admit it was probably this comment that most convinced me that I had to at least give it a try.)

lacostenycoder commented 3 years ago

I'm trying to run elasticsearch@6 on MacOS Big Sur and I can't get it to work. It installs but I see this:

Name            Status  User  Plist
elasticsearch@6 error   lance /Users/lance/Library/LaunchAgents/homebrew.mxcl.elasticsearch@6.plist
influxdb        started lance /Users/lance/Library/LaunchAgents/homebrew.mxcl.influxdb.plist
postgresql@9.5  started lance /Users/lance/Library/LaunchAgents/homebrew.mxcl.postgresql@9.5.plist
redis           started lance /Users/lance/Library/LaunchAgents/homebrew.mxcl.redis.plist

Is there anyway I can get it to work before this issue is fixed? This is a dependency required to develop for the company I work for.

jonchang commented 3 years ago

Please open a new issue for this, thanks.

Moisan commented 3 years ago

I think predictionio should be disabled since its webpage states

Project Predictionio has retired. For details please refer to its Attic page.

The attic page says

Apache PredictionIO moved into the Attic in Sep 2020. [...] The website, downloads and issue tracker all remain open, though the issue tracker is read-only.

carlocab commented 3 years ago

Good reason to deprecate it then.

Moisan commented 2 years ago

visp and fdroidserver are now fixed:

> brew deps visp  | grep vtk
vtk
> brew deps fdroidserver | grep python
python@3.9
carlocab commented 2 years ago

We should remove the ones that no longer need it from audit_exceptions/versioned_dependencies_conflicts_allowlist.json.

carlocab commented 2 years ago

Adding watchman to the allowlist in #89256, and coin3d, gjs, pyside and pyside@2 in #89262. I don't think we'll ever finish the Python 3.10 migration otherwise.

iMichka commented 2 years ago

I removed the autogen dep from gnutls. I think there were more cleanups than this, but anyway anjuta is fine now. Is there anything else to do here now, as the initial list has been tackled.

cho-m commented 2 years ago

From current entries in allowlist:

Everything else seems to be for Python 3.9 to 3.10 migration. May want track that migration in separate work item rather than keeping this open, which has been open for over a year now.

Beyond that, we are stuck allowlist entries until hive is updated and maybe deprecated formulae are disabled/removed.

EDIT: Dropping help wanted for now until there is actionable items available.

github-actions[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.