Homebrew / legacy-homebrew

💀 The former home of Homebrew/homebrew (deprecated)
https://brew.sh
27.02k stars 11.37k forks source link

Error: No such file or directory - /usr/local/Cellar/progress #45483

Closed urdh closed 7 years ago

urdh commented 8 years ago

This happened today when I tried running brew upgrade:

ssimon@lagrange ~> brew upgrade
Updated Homebrew from 513cbf03 to 61c6ea3a.
Updated 6 taps (caskroom/cask, homebrew/fuse, homebrew/games, homebrew/headonly, homebrew/science, homebrew/versions).
==> New Formulae
dfmt                                     libstatgrab
gdub                                     mailhog
glslang                                  shairport-sync
homebrew/headonly/depot-tools            sqliteodbc
==> Updated Formulae
aces_container                           kafkacat
akka                                     keybase (installed)
antlr                                    khard
app-engine-go-32                         ldc (installed)
app-engine-go-64                         libebml
ats2-postiats                            libftdi
autocode                                 libmatroska
aws-elasticbeanstalk                     libsodium
awscli                                   libuecc
bear                                     libxml++
boot-clj                                 mackup
cask                                     magit
ceylon                                   mesos
charm-tools                              mongrel2
diff-pdf (installed)                     mosquitto
dpkg                                     mpd
duck                                     nghttp2
embulk                                   nsd
emscripten                               osxfuse
evince                                   parallel
fibjs                                    passenger
flow                                     pdf2htmlex
git-lfs                                  pkg-config (installed)
git-plus                                 plantuml
gor                                      poppler (installed)
grails                                   pre-commit
grib-api                                 purescript
groonga                                  pygobject3
homebank                                 pyqt5
homebrew/games/pioneer                   qbs
homebrew/science/cantera                 quassel
homebrew/science/flint                   rbenv-bundler-ruby-version
homebrew/science/hypre                   ruby-build
homebrew/science/mlst                    sickrage
homebrew/science/nextflow                snag
homebrew/science/p4est                   sshuttle
homebrew/science/petsc                   sstp-client
homebrew/science/qr_mumps                swiftgen
homebrew/science/snp_sites               task
homebrew/science/sundials                telegraf
homebrew/science/wcstools                terraform
homebrew/versions/glfw3                  tika
ios-webkit-debug-proxy                   vault
ircd-hybrid                              vdirsyncer
jenkins                                  vegeta
jruby                                    xctool (installed)
jsonnet                                  xonsh
juju                                     yle-dl
jvgrep                                   youtube-dl
Error: No such file or directory - /usr/local/Cellar/progress

Not surprisingly, installing progress makes the problem go away:

ssimon@lagrange ~> brew install progress
==> Downloading https://homebrew.bintray.com/bottles/progress-0.9.el_capitan.bottle.tar.gz
######################################################################## 100,0%
==> Pouring progress-0.9.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/progress/0.9: 5 files, 72K
ssimon@lagrange ~> brew upgrade
==> Upgrading 36 outdated packages, with result:
aria2 1.19.2, at-spi2-atk 2.18.1, at-spi2-core 2.18.1, caskroom/cask/brew-cask 0.59.0, progress HEAD, diff-pdf 0.2_4, ffmpeg 2.8.1_1, gdal 1.11.3, git 2.6.2, gobject-introspection 1.46.0, gsettings-desktop-schemas 3.18.1, gtk+ 2.24.28_3, gtk+3 3.18.2, harfbuzz 1.0.4, heroku-toolbelt 3.42.20, imagemagick 6.9.2-4, keybase 0.8.23, lcov 1.12, ldc 0.16.1, libressl 2.3.0, mbedtls 2.1.1, node 4.2.1, pango 1.38.1, peco 0.3.4, pixman 0.32.8, pkg-config 0.29, poppler 0.37.0, postgresql 9.4.5, qt 4.8.7_1, rebar 2.6.1, scons 2.4.0, sqlite 3.9.1, tmux 2.1, x264 r2601, xctool 0.2.7, xz 5.2.2
...

Is this just a new dependency Homebrew can't automatically resolve, or what's going on?

Brew doctor

ssimon@lagrange ~> brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry and just ignore them. Thanks!

Warning: Your XQuartz (2.7.7) is outdated
Please install XQuartz 2.7.8:
  https://xquartz.macosforge.org

Warning: You have unlinked kegs in your Cellar
Leaving kegs unlinked can lead to build-trouble and cause brews that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
    at-spi2-atk
    pkg-config
    python

Warning: Your Xcode (7.0) is outdated
Please update to Xcode 7.1.
Xcode can be updated from the App Store.

System information

ssimon@lagrange ~> system_profiler SPSoftwareDataType
Software:

    System Software Overview:

      System Version: OS X 10.11 (15A278b)
      Kernel Version: Darwin 15.0.0
      Boot Volume: Joseph Louis Lagrange
      Boot Mode: Normal
      Computer Name: Joseph Louis Lagrange
      User Name: Simon Sigurdhsson (ssimon)
      Secure Virtual Memory: Enabled
      Time since boot: 2 days 22:34
bfontaine commented 8 years ago

progress is the new name of the cv formula; it might have something to do here. cc @vladshablinsky

vladshablinsky commented 8 years ago

@urdh I suppose it's not brew upgrade, because upgrade doesn't update anything. However I tried to install formula and then update it and run brew upgrade. The result was as I expected -- there was an error saying the migration was needed, which is OK.

Suppose it's the update you're talking about. I don't see any information neither about cv nor about progress here (UPD: formula names are in alphabetic order, so this remark has sense here):

dfmt                                     libstatgrab
gdub                                     mailhog
glslang                                  shairport-sync
homebrew/headonly/depot-tools            sqliteodbc
==> Updated Formulae
aces_container                           kafkacat
akka                                     keybase (installed)
antlr                                    khard
app-engine-go-32                         ldc (installed)
app-engine-go-64                         libebml
ats2-postiats                            libftdi
autocode                                 libmatroska
aws-elasticbeanstalk                     libsodium
awscli                                   libuecc
bear                                     libxml++
boot-clj                                 mackup
cask                                     magit
ceylon                                   mesos
charm-tools                              mongrel2
diff-pdf (installed)                     mosquitto
dpkg                                     mpd
duck                                     nghttp2
embulk                                   nsd
emscripten                               osxfuse
evince                                   parallel
fibjs                                    passenger
flow                                     pdf2htmlex
git-lfs                                  pkg-config (installed)
git-plus                                 plantuml
gor                                      poppler (installed)
grails                                   pre-commit
grib-api                                 purescript
groonga                                  pygobject3
homebank                                 pyqt5
homebrew/games/pioneer                   qbs
homebrew/science/cantera                 quassel
homebrew/science/flint                   rbenv-bundler-ruby-version
homebrew/science/hypre                   ruby-build
homebrew/science/mlst                    sickrage
homebrew/science/nextflow                snag
homebrew/science/p4est                   sshuttle
homebrew/science/petsc                   sstp-client
homebrew/science/qr_mumps                swiftgen
homebrew/science/snp_sites               task
homebrew/science/sundials                telegraf
homebrew/science/wcstools                terraform
homebrew/versions/glfw3                  tika
ios-webkit-debug-proxy                   vault
ircd-hybrid                              vdirsyncer
jenkins                                  vegeta
jruby                                    xctool (installed)
jsonnet                                  xonsh
juju                                     yle-dl
jvgrep                                   youtube-dl

that makes things more difficult. All the migrations happen before making the report, so if the problem was in migrations themself, we wouldn't see the report message.

The last thing done after dumping the report is Descriptions.update_cache(report). I took a look at description.rb and I can't be sure that it causes an error. There are some lines inserted on October 9th and the others are inserted on May 5th, so I'm not sure.

It's hard to say for me what caused your error, it looks strange. If you could provide some more information like reproducing the error running your commands with --verbose and --debug options it would be easier to find a bug here.

urdh commented 8 years ago

Yes, it's brew update (but it happened for subsequent brew upgrade as well). Unfortunately I can't reproduce the issue anymore. (I tried passing the -v flag to brew upgrade earlier but it printed nothing other than the error, IIRC.)

One thing I noticed as well is that when running brew upgrade after initially running brew install progress, Homebrew installs the HEAD version even though the installed version was not.

I think that I originally had installed cv from homebrew/headonly. Maybe there's an edge case with moving and renaming formulae?

vladshablinsky commented 8 years ago

I think that I originally had installed cv from homebrew/headonly. Maybe there's an edge case with moving and renaming formulae?

@urdh there shouldn't be, because cases like this were tested, but I'll try to test tomorrow anyway. I should say again it's strange that you have the error after dump. Thanks.

vladshablinsky commented 8 years ago

I think that I originally had installed cv from homebrew/headonly. Maybe there's an edge case with moving and renaming formulae?

@urdh thanks for this information, it helped.

I found the bug. You can reproduce it by doing these steps:

cd /usr/local/Library/Homebrew
brew tap homebrew/headonly --full
cd ../Taps/homebrew/homebrew-headonly
git reset --hard fa8d726
brew uninstall cv
brew install --HEAD homebrew/headonly/cv
cd /usr/local/Library/Homebrew
git checkout master
git reset --hard 513cbf03
brew update
brew upgrade cv

Why do we encounter the error? The reason is the following. We removed formula from tap, so we can't instantiate it. So, when we try to get formula for cv we try to get it from tap, then we fail, because we deleted formula from tap, so we try to instantiate it from core, and when we try it we end up with wrong formula progress, because it's the new name for cv from core.

First step is Formula.installed

# formula.rb
# ...
 def self.installed
    @installed ||= racks.map do |rack|
      begin
        Formulary.from_rack(rack)  # <---------------- WE GO HERE
      rescue FormulaUnavailableError, TapFormulaAmbiguityError, TapFormulaWithOldnameAmbiguityError
      end
    end.compact
  end
# ...

Then we instantiate formula

# formulary.rb
# Formulary.from_rack
# ...
      begin
        factory("#{tap}/#{rack.basename}", spec)
      rescue FormulaUnavailableError
        # formula may be migrated to different tap. Try to search in core and all taps.
        factory(rack.basename.to_s, spec)  # <---------------- WE GO HERE
      end
# ...

Then we get outdated brews and do the following check inside it before we call formula.rack.subdirs for that wrongly instantiated formula.

# cmd/outdated.rb
# outdated_brews
# ...
      if f.oldname && !f.rack.exist? && (dir = HOMEBREW_CELLAR/f.oldname).directory? &&
        !dir.subdirs.empty? && f.tap == Tab.for_keg(dir.subdirs.first).tap
        raise Migrator::MigrationNeededError.new(f)
      end
# ...

The check passes, because of f.tap == Tab.for_keg(dir.subdirs.first).tap. (core != headonly) Then we fail.

The problem here is with deletion and with wrong formula that we instantiate.

@mikemcquaid @xu-cheng any thoughts here?

vladshablinsky commented 8 years ago

The easiest solution that comes to mind is to have something like "name" => "", in formula_renames and handle this case somehow. But the behaviour of renaming process is going to be changed soon, so maybe it doesn't worth the effort to fix that bug right now, because it's pretty rare, and will be fixed soon anyway I suppose.

To reproduce it we need:

  1. install formula from tap
  2. rename formula with the same name in core
  3. delete formula from tap.
MikeMcQuaid commented 8 years ago

@vladshablinsky I'm happy to have that as a workaround and document that in moving things from taps. If it'll be fixed soon by something else though: that seems fine.

vladshablinsky commented 8 years ago

@mikemcquaid I was thinking about something like this: https://github.com/Homebrew/homebrew/compare/master...vladshablinsky:rename-remove . Pretty easy, pretty hacky. Seems that it solves the problem.

Maybe it's better to have something like this or don't have anything at all. I think the same problem can appear when we try instantiating formula using aliases, I mean wrong one is instantiated. I'm not sure, but I'll test.

Note about formula deletions: I said that I would continue working on formula renames and I think the deletions will be the part of the feature. You might have noticed that ~15 days left and there is still no progress, but I'm trying hard to begin ASAP.

MikeMcQuaid commented 8 years ago

@vladshablinsky Seems fine to me 👍

I said that I would continue working on formula renames and I think the deletions will be the part of the feature. You might have noticed that ~15 days left and there is still no progress, but I'm trying hard to begin ASAP.

Don't worry, we don't pay you so take your time :wink:

tdsmith commented 8 years ago

@vladshablinsky Do you know if you landed something that fixed this? I'm tempted to close this anyway since it seems isolated and resolved.

vladshablinsky commented 8 years ago

@tdsmith it will be fixed with updated formula renames implementation so deletions will be supported. I'm working on it now.

MikeMcQuaid commented 7 years ago

We'll accept PRs for this but we're not actively working on it at this time.