Upgrading osgeo-qgis-res to 3.8.0 fails from bottle as well as source on High Sierra

kevinstadler commented 4 years ago

Please answers the following

I've...

I read
- [X] the reporting issues
- [X] the troubleshooting
[X] I checked the documentation and found no answer
[X] I tried to install after following the suggestions
[X] I am running the latest version (brew update twice?)
[x] I checked to make sure that this issue has not already been filed
[X] I uploaded logs: brew gist-logs formula

Describe the bug

After all other dependencies have already been upgraded, brew upgrade osgeo-qgis-res fails:

downloading the High Sierra bottle produces a 404: checking the bottle download location under https://bottle.download.osgeo.org there is only a osgeo-qgis-res-3.8.0.high_sierra.bottle.1.tar.gz (note the .1 part), not the osgeo-qgis-res-3.8.0.high_sierra.bottle.tar.gz that the Formula is trying to download
building from source fails during installation of the optional blosc resource (full log below)

I even tried manually downloading the osgeo-qgis-res-3.8.0.high_sierra.bottle.1.tar.gz bottle and putting it in the appropriate download cache path at brew --cache --force-bottle osgeo-qgis-res, but an installation attempt from there also fails because of the SHA sum mismatch:

$ brew upgrade osgeo-qgis-res
Updating Homebrew...
==> Upgrading 1 outdated package:
osgeo/osgeo4mac/osgeo-qgis-res 3.6.3_1 -> 3.8.0
==> Upgrading osgeo/osgeo4mac/osgeo-qgis-res 3.6.3_1 -> 3.8.0 
==> Downloading https://bottle.download.osgeo.org/osgeo-qgis-res-3.8.0.high_sierra.bottle.tar.gz
Already downloaded: /Users/kevin/Library/Caches/Homebrew/downloads/582b0b8d211685ac2836da169b41d7584f4492bd8d7d8e401f3997f86915b1c8--osgeo-qgis-res-3.8.0.high_sierra.bottle.tar.gz
Error: SHA256 mismatch
Expected: 8e9ab749b7f1aa6d8e423803ac837038cc46c1c63e9b5c6581fa80068a7fc0c4
  Actual: f8ae7bd76b57eca4f8309d654f8bda9db1904079f076cdeed4a7ccb16b9eccd0
 Archive: /Users/kevin/Library/Caches/Homebrew/downloads/582b0b8d211685ac2836da169b41d7584f4492bd8d7d8e401f3997f86915b1c8--osgeo-qgis-res-3.8.0.high_sierra.bottle.tar.gz
To retry an incomplete download, remove the file above.
Warning: Bottle installation failed: building from source.
...

In the end I was able to install from the bottle file by changing the SHA sum in the Formula file. The only broken dependency when trying to start up QGIS was:

dyld: Library not loaded: /usr/local/opt/spatialindex/lib/libspatialindex.5.dylib
  Referenced from: /usr/local/Cellar/osgeo-qgis/3.8.0_1/QGIS.app/Contents/MacOS/./QGIS
  Reason: image not found
Abort trap: 6

which I fixed using

$ cd /usr/local/opt/spatialindex/lib/
$ ln -s libspatialindex.6.1.1.dylib libspatialindex.5.dylib

To Reproduce

brew upgrade osgeo-qgis-res

Expected behavior

osgeo-qgis-res should be installed from the 3.8.0 bottle.

Additional context

Running High Sierra 10.13.6 on a 2012 Macbook Air, I previously had osgeo-qgis-res 3.6.3_1 installed.

brew doctor output:

$ brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: You have unlinked kegs in your Cellar.
Leaving kegs unlinked can lead to build-trouble and cause brews that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
  proj
  postgresql
  w3m
  ghostscript
  gdal
  libgeotiff
  libspatialite
  postgis
  osgeo-postgresql
  netcdf
  osgeo-postgresql@10

Warning: Homebrew's sbin was not found in your PATH but you have installed
formulae that put executables in /usr/local/sbin.
Consider setting the PATH for example like so:
  echo 'export PATH="/usr/local/sbin:$PATH"' >> ~/.bash_profile

Warning: Some installed formulae are missing dependencies.
You should `brew install` the missing dependencies:
  brew install apr apr-util asio byacc castxml ceres-solver cppunit dcmtk fftw freeglut gflags glfw glog gpsbabel gstreamer gtkglext lbzip2 lighttpd llvm mesa metis minizip mysql numpy@1.16 opencv opencv@2 osgeo-grass osgeo-insighttoolkit osgeo-lastools osgeo-opencollada osgeo-openscenegraph osgeo-orfeo osgeo-ossim osgeo-qt-mysql osgeo-qt-odbc osgeo-qt-psql osgeo-saga-lts osgeo-taudem osgeo-whitebox-tools pyqt-qt4 podofo potrace pth regex-opt spawn-fcgi subversion suite-sparse tinyxml utf8proc v8

Run `brew missing` for more details.

Note that the unlinked libraries above are the default homebrew ones, which are exactly unlinked because the osgeo- prefixed ones are linked in their place.

Logs

https://gist.github.com/645551da0077623a134b94dc015b4815

Complete log of the failed brew upgrade osgeo-qgis-res:

$ brew upgrade
Updating Homebrew...
Successfully rebased and updated refs/heads/master.
==> Auto-updated Homebrew!
Updated 1 tap (homebrew/cask).
==> Updated Casks
sip

Error: Not upgrading 2 pinned packages:
libtensorflow 2.1.0_1, osmium-tool 1.11.1
==> Upgrading 1 outdated package:
osgeo/osgeo4mac/osgeo-qgis-res 3.6.3_1 -> 3.8.0
==> Upgrading osgeo/osgeo4mac/osgeo-qgis-res 3.6.3_1 -> 3.8.0 
==> Downloading https://bottle.download.osgeo.org/osgeo-qgis-res-3.8.0.high_sierra.bottle.tar.gz

curl: (22) The requested URL returned error: 404 Not Found
Error: Failed to download resource "osgeo-qgis-res"
Download failed: https://bottle.download.osgeo.org/osgeo-qgis-res-3.8.0.high_sierra.bottle.tar.gz
Warning: Bottle installation failed: building from source.
==> Downloading https://gist.githubusercontent.com/dakcarto/11385561/raw/e49f75ecec96ed7d6d3950f45ad3f30fe94d4fb2/pyqgis_startup.py
Already downloaded: /Users/kevin/Library/Caches/Homebrew/downloads/056c5e51de1c0948e4cccaf98c1e0fa121e399a819dd0977c4a4e56fb260c626--pyqgis_startup.py
==> Downloading https://files.pythonhosted.org/packages/b1/72/2d70c5a1de409ceb3a27ff2ec007ecdd5cc52239e7c74990e32af57affe9/virtualenv-15.2.0.tar.gz
Already downloaded: /Users/kevin/Library/Caches/Homebrew/downloads/e390edd594d1668ad899868951573cc02842e8f82003ec75bd388377d3a3f373--virtualenv-15.2.0.tar.gz
==> /usr/local/opt/python/bin/python3 -c import setuptools... --no-user-cfg install --prefix=/private/tmp/osgeo-qgis-res--homebrew-virtualenv-20200329-54634-1io2vgs/target --install-scri
==> /usr/local/opt/python/bin/python3 -s /private/tmp/osgeo-qgis-res--homebrew-virtualenv-20200329-54634-1io2vgs/target/bin/virtualenv -p /usr/local/opt/python/bin/python3 /usr/local/Cel
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip3 install --upgrade -v setuptools pip wheel
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed rpy2
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed sphinxcontrib-websupport
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip3 install --upgrade -v setuptools pip==9.0.3 wheel
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed pyRscript
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip3 install --upgrade -v setuptools pip<19.0.0 wheel
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed requests
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed six
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed future
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed Sphinx
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed setuptools-scm
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed chardet
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed idna
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed urllib3
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed PySocks
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed Pillow
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed cycler
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed kiwisolver
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed tornado
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed Unidecode
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed pyparsing
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed MarkupSafe
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed nose
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed Cython
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed python-dateutil
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed pytz
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed Jinja2
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed OWSLib
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed psycopg2
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed Pygments
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed PyYAML
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed dbus-python
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed PyOpenGL
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed certifi
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed funcsigs
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed coverage
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed mock
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed pbr
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed termcolor
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed oauthlib
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed pyOpenSSL
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed httplib2
==> Downloading https://files.pythonhosted.org/packages/93/48/956b9dcdddfcedb1705839280e02cbfeb2861ed5d7f59241210530867d5b/numpy-1.16.3.zip
######################################################################## 100.0%
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/python3 setup.py build --fcompiler=gnu95 --parallel=4 install --prefix=/usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor
==> Downloading https://files.pythonhosted.org/packages/cb/97/361c8c6ceb3eb765371a702ea873ff2fe112fa40073e7d2b8199db8eb56e/scipy-1.3.0.tar.gz
######################################################################## 100.0%
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/python3 setup.py build --fcompiler=gnu95 install --prefix=/usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor
==> Downloading https://files.pythonhosted.org/packages/26/04/8b381d5b166508cc258632b225adbafec49bbe69aa9a4fa1f1b461428313/matplotlib-3.0.3.tar.gz
######################################################################## 100.0%
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/python3 setup.py install --prefix=/usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed argparse
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed asn1crypto
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed atlas
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed backports.functools_lru_cache
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed beautifulsoup4
==> /usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/pip install -v --no-deps --no-binary :all: --ignore-installed blosc
Last 15 lines from /Users/kevin/Library/Logs/Homebrew/osgeo-qgis-res/53.pip:
  File "/usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/lib/python3.7/site-packages/pip/_internal/resolve.py", line 256, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/lib/python3.7/site-packages/pip/_internal/resolve.py", line 209, in _get_abstract_dist_for
    self.require_hashes
  File "/usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 298, in prepare_linked_requirement
    abstract_dist.prep_for_dist(finder, self.build_isolation)
  File "/usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 109, in prep_for_dist
    "Installing build dependencies"
  File "/usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/lib/python3.7/site-packages/pip/_internal/build_env.py", line 122, in install_requirements
    call_subprocess(args, show_stdout=False, spinner=spinner)
  File "/usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/lib/python3.7/site-packages/pip/_internal/utils/misc.py", line 705, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip._internal.exceptions.InstallationError: Command "/usr/local/Cellar/osgeo-qgis-res/3.8.0/libexec/vendor/bin/python3 -m pip install --ignore-installed --no-user --prefix /private/tmp/pip-build-env-kn8fn1w2 --no-warn-script-location -v --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel scikit-build cmake ninja" failed with error code 1 in None
You are using pip version 18.1, however version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

If reporting this issue please do so at (not Homebrew/brew or Homebrew/core):
  https://github.com/osgeo/homebrew-osgeo4mac/issues

==> Checking for dependents of upgraded formulae...
==> No dependents found

alazarolop commented 4 years ago

Hi @kevinstadler, at this moment, osgeo-qgis formula is broken and it will fail to install. The suggested way to use QGIS in macOS currently is the cask version brew cask install qgis https://github.com/OSGeo/homebrew-osgeo4mac/issues/1362#issuecomment-596455760

flowrean commented 4 years ago

@alazarolop Is the formula currently still broken? I am unable to install QGIS after a lot of tries.

$ brew tap osgeo/osgeo4mac
$ brew install osgeo-qgis
Error: No available formula with the name "osgeo-qgis" 
==> Searching for a previously deleted formula (in the last month)...
Warning: homebrew/core is shallow clone. To get complete history run:
  git -C "$(brew --repo homebrew/core)" fetch --unshallow

Error: No previously deleted formula found.
==> Searching for similarly named formulae...
Error: No similarly named formulae found.
==> Searching taps...
==> Searching taps on GitHub...
Error: No formulae found in taps.

alazarolop commented 4 years ago

@flowrean the formula is no longer available in the tap. Instead, the all-in-one formula "cask" is the recommended way:

brew cask install qgis

A new packaging method with GDAL 3 and PROJ >6 is under development: https://github.com/qgis/QGIS-Mac-Packager

flowrean commented 4 years ago

@alazarolop Thanks, using cask worked! Just to understand things better, could you elaborate on the relation between osgeo4mac, the cask and QGIS-Mac-Packager? Would the new packager eventually be used in the cask, or replace it in Homebrew? Is there any difference between the cask app and the app installed through a DMG from download.qgis.org?

alazarolop commented 4 years ago

Good!

QGIS-Mac-Packager is the repo to create the DMG that you can downloaded either from the webpage or using the cask functionality. The new package system will eventually replace the current version which is based on osgeo4mac. This one, Osgeo4mac, is an independent Homebrew tap maintained by the community and it's also under revision.

OSGeo / homebrew-osgeo4mac

Upgrading osgeo-qgis-res to 3.8.0 fails from bottle as well as source on High Sierra #1369