Open luispuerto opened 5 years ago
To take into consideration: https://github.com/Homebrew/homebrew-core/issues/31510
I think we should also check what are they take into duplicate formulae and so. Either we ask in the issue tracker of core or here: https://discourse.brew.sh
Very timely reflection.
Beyond the difficulties involved, I especially agree with
"We need to keep the number of formulae in the tap as low as possible."
Now, if this is the official tap of the Osgeo project, this tap must be the host of the formulas related to it
On the duplicity of formulas.
What options are not available in the core formulas? Why are they not available? What advantages do these options offer? It is worth having the formulas dulicadas to have these options, or can we wait for the problems that now prevent them to be active in the nucleus to be solved?
Sorry for the inconvenience. But know understand that we decided to rename formulas to make other things simpler. The process took time due to the restrictions in CI and the delay of Homebrew Core in updating many of the dependencies, for example Qt and PROJ (a few hours ago). This leads to rebuild all the bottles. For this reason and another is that we were working on optimizing the tap for the future.
p/d: I will be rebuilding everything for PROJ 6.
we are working to migrate the bottles to osgeo, bintray already caused many problems.
We need to keep the number of formulae in the tap as low as possible.
We should try to rely as much as possible into core formulae.
Yes please! I currently use osgeo4mac to install qgis. Its 294 dependencies include:
osgeo-gdal osgeo-gdal-python osgeo-hdf4 osgeo-hexer osgeo-laszip@2 osgeo-laz-perf osgeo-libgeotiff osgeo-libkml osgeo-liblas osgeo-libspatialite osgeo-matplotlib osgeo-netcdf osgeo-opencollada osgeo-openscenegraph osgeo-pcl osgeo-pdal osgeo-postgresql osgeo-proj osgeo-pyqt osgeo-qt-webkit osgeo-sip osgeo-vtk
This is quite a lot. Why does qgis want to overwrite the core postgresql, and why does qgis have a hard requirement on postgresql anyway?
Installing qgis also installs lighttpd, mysql, openldap, gtk, unbound and e2fsprogs. I don't understand why, especially the last one.
@grischard For several reasons. Mainly to not have as many flags in the formulas and to provide more supports. In this way the user will not have to build on his machine if he requires optional supports (but required by many).
Also to have more control of when to update, there are many dependencies that are updated very late and inopportunely. And that they do not provide all their functions.
For example, for PostgreSQL the versions of homebrew-core do not work for a migration process.
We are having problems with Bintray, that's why the work stopped. Tonight we will remate everything.
p/d: yes, e2fsprogs can be removed, since it is for linux.
I'm really thankful for the qgis packages, but I feel misunderstood as an user. I don't get control of when to update, I'm married to osgeo4mac's postgresql and proj and gdal and so on. I'm currently sitting on a broken osgeo4mac qgis that can't find libproj.13.dylib in osgeo4mac's proj, and wondering if installing all these osgeo formulae won't just break something else.
Yeah, I got bitten by that postgresql migration thing myself too, but isn't the right way to go about this to fix the core?
Can qgis be built with, e.g., mysql support but not include the whole server as a dependency?
All secondary dependencies will be on the tap. For this reason, everything will work correctly.
I'm already working to load everything again http://bottle.download.osgeo.org/
Once updated you will realize that everything works very well.
to temporarily fix proj: ln -s libproj.15.dylib libproj.13.dylib
Yup, that gets qgis to load, thanks.
I see what you're trying to do. But the way I had to symlink libproj.15.dylib to libproj.13.dylib, the way PyQt5.QtCore seems to be missing as a dependency which produces an error when opening, the way python produces a second error when opening because it can't find the module called 'qgis', scares me.
Just wait, it will be a few hours and everything will return to normal.
I really regret the inconvenience, but we had problems with Bintray in the middle of the tap update process.
I'll be patient then. Thank you!
I have finished configuring everything! I will start to migrate the bottles from Bintray and update the missing ones.
@grischard I really don't know what is your specific problem, but as thing are right now, you can make it work qgis with a little bit of tweaking. At least it's working for me.
Anyhow.... wait a little bit and probably in the next hours everything is going to back to normal.
During the morning I will transfer the bottles that are in bintray to osgeo.org.
After eating I will sit down to complete the updates.
Everything will return to normal, in the future we will not have these problems.
I have already built qgis on my machine, I just need to fix something in orfeo. Later I apply all the changes.
already building qgis! #1063
the bottles for qgis are ready!
Thank you so much for your hard work!
I just nuked my previous installation of qgis + dependencies and reinstalled from scratch, everything looks good.
Thank you. So I can now nuke any unlinked foo
package where qgis required I install osgeo-foo
?
@grischard yes, we will use our versions for some dependencies.
Yeah!!! @fjperini thanks a lot for all the work these last weeks, we have a really great version of QGIS up and running thanks to you!
I'm not sure how things are supposed to be installed cleanly now. QGIS depends on ffmpeg (why?) which depends on openjpeg, which conflicts with osgeo-insighttoolkit on /usr/local/lib/pkgconfig/libopenjp2.pc. My osm2pgsql requires, directly or indirectly, postgres, gdal and proj, and qgis installs its own version. All this for QGIS. Hmm.
Is the long-term plan to have qgis in the main homebrew repository?
Is the long-term plan to have qgis in the main homebrew repository?
No, this is in anyway possible now with the current Homebrew core policy... QGIS is an app so there is no place for it in Core. It has to be build and has several dependencies... so it doesn't fit either in Cask.
I know that it's annoying but it's what it's
I'm not sure how things are supposed to be installed cleanly now. QGIS depends on ffmpeg (why?) which depends on openjpeg, which conflicts with osgeo-insighttoolkit on /usr/local/lib/pkgconfig/libopenjp2.pc. My osm2pgsql requires, directly or indirectly, postgres, gdal and proj, and qgis installs its own version. All this for QGIS. Hmm.
I think, @fjperini can give you a more comprehensive answer about it since he has engineered and knitted the formulae... However, I can give you some insights:
brew deps --tree osgeo-qgis
to see it. We are really open so suggestions, feedback and even more to people to participate in the tap. In other words, help is wanted!
PS/ in the opinion of some of us, we have to best QGIS in years using Homebrew, and I would say that it's even better than the one you can install from the official installer. Thanks to @fjperini we don't have to build anymore in our machines to have most the capabilities. This doesn't mean that it's perfect, just better. We want to make it even better 😉 and easier, but QGIS is complicated.
It's definitely a lot better than all the other methods we had before!
Hi @grischard!
The idea is to have a QGIS as complete as possible. For this reason so many dependencies are installed, ffmpeg
corresponds to osgeo-grass
.
Having other used formulas in the repository allows us to correct conflicts between dependencies and other problems. We could add osm2pgsql
to the repository.
I just had a look at the casks policy, and couldn't find anything. It seems to be the opposite actually: mactex
happily has depends_on formula: 'ghostscript'
.
@fjperini yeah but do you really want to be maintaining all these formulas for just a few dependencies?
I'd be interested to know how you're handling dependencies on your installations. Currently, I can't link netcdf
because it conflicts with osgeo-netcdf
, and I can't remove netcdf
either because it is required by osgeo-taudem
, which I'll never use but qgis
requires. Scratching my chin looking at my terminal.
@grischard In my installation I have not had any problems with the dependencies. I could help you if you have problems with osgeo-netcdf
.
We could choose to have a smaller version and a more complete version, something like a osgeo-qgis-suite
.
Having a complete qgis is necessary, a large part of the community uses many of the tools and building everything requires time and knowledge about the problems that may arise.
I just had a look at the casks policy, and couldn't find anything. It seems to be the opposite actually: mactex happily has depends_on formula: 'ghostscript'.
The problem isn't exactly that... although you are going to hardly find a formula there with the amount of dependencies QGIS has. The problem is, you need to have the build done beforehand since cask isn't building just installing. In other words, the cask tap is for example for the install file you can find in the QGIS webside.
Why isn't there then a cask with that QGIS install? Well... there was for a while I think, or they try do have one, but I think the QGIS installer is too complicated for casks. (yes it's complicated)
You can know more here: https://github.com/Homebrew/homebrew-cask/pull/41883
QGIS is a conundrum, mainly because dependencies.
I've tried to fix my qgis installation. basically what I wanted to do was remove it all. Since it messes with my brew
I think part of the problem is that I have postgressql
& gdal
for other things as well.
this is the current output of brew doctor
(It looked a lot worse before):
Warning: You have unlinked kegs in your Cellar.
Leaving kegs unlinked can lead to build-trouble and cause brews that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
proj
libgeotiff
libspatialite
netcdf
open-scene-graph
Warning: Some installed formulae are missing dependencies.
You should `brew install` the missing dependencies:
brew install osgeo-pyqt osgeo-sip
Run `brew missing` for more details.
~ » brew missing
osgeo-pyqt-webkit: osgeo-sip osgeo-pyqt
osgeo-qgis-res: osgeo-sip osgeo-pyqt
Do you have any kind of guide as to how I would fix this due to your renamings and whatnot. What should II merge with what etc and is it meant to be done on all formulas independently?
Thank you in advance!
@LeonardAukea you can start by uninstalling these dependencies:
proj
libgeotiff
libspatialite
netcdf
open-scene-graph
and install these:
osgeo-pyqt-webkit
osgeo-sip
osgeo-pyqt
@fjperini Ok, thanks. this is what sort of happens:
~ » brew uninstall proj 00:03:24
Error: Refusing to uninstall /usr/local/Cellar/proj/6.0.0
because it is required by gdal, libgeotiff, liblas, libspatialite, osgeo-gdal-python, osgeo-grass, osgeo-liblas, osgeo-orfeo, osgeo-ossim, osgeo-pyspatialite, osgeo-qgis-res, osgeo-saga-lts, osgeo-taudem and pdal, which are currently installed.
You can override this and force removal with:
brew uninstall --ignore-dependencies proj
~ » 00:03:36
~ » brew uninstall --ignore-dependencies proj 00:03:48
Uninstalling /usr/local/Cellar/proj/6.0.0... (56 files, 13MB)
~ » brew uninstall --ignore-dependencies libgeotiff 00:04:01
Uninstalling /usr/local/Cellar/libgeotiff/1.5.1... (39 files, 576.9KB)
~ » brew uninstall --ignore-dependencies libspatialite 00:04:11
Uninstalling /usr/local/Cellar/libspatialite/4.3.0a_7... (31 files, 18.6MB)
~ » brew uninstall --ignore-dependencies netcdf 00:04:23
Uninstalling /usr/local/Cellar/netcdf/4.6.3... (85 files, 6.2MB)
~ » brew uninstall --ignore-dependencies open-scene-graph 00:04:49
Uninstalling /usr/local/Cellar/open-scene-graph/3.6.3... (8,893 files, 229.2MB)
~ » brew install osgeo-pyqt-webkit 00:06:20
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 1 tap (brewsci/bio).
==> Updated Formulae
brewsci/bio/minimap2
Warning: osgeo/osgeo4mac/osgeo-pyqt-webkit 5.12.1 is already installed and up-to-date
To reinstall 5.12.1, run `brew reinstall osgeo-pyqt-webkit`
~ » brew reinstall osgeo-pyqt-webkit 00:06:56
==> Reinstalling osgeo/osgeo4mac/osgeo-pyqt-webkit
==> Installing dependencies for osgeo/osgeo4mac/osgeo-pyqt-webkit: osgeo-sip and osgeo-pyqt
==> Installing osgeo/osgeo4mac/osgeo-pyqt-webkit dependency: osgeo-sip
==> Downloading https://bottle.download.osgeo.org/osgeo-sip-4.19.15.mojave.bottle.1.tar.gz
curl: (22) The requested URL returned error: 502 Bad Gateway
Error: Failed to download resource "osgeo-sip"
Download failed: https://bottle.download.osgeo.org/osgeo-sip-4.19.15.mojave.bottle.1.tar.gz
Warning: Bottle installation failed: building from source.
==> Downloading https://www.riverbankcomputing.com/static/Downloads/sip/sip-4.19.15.tar.gz
curl: (22) The requested URL returned error: 404 Not Found
Error: An exception occurred within a child process:
DownloadError: Failed to download resource "osgeo-sip"
Download failed: https://www.riverbankcomputing.com/static/Downloads/sip/sip-4.19.15.tar.gz
I've got similar url error codes earlier.
@LeonardAukea solved the problem with osgeo-sip
#1102
@fjperini Thanks. but something seems to be messy still. Looks like the packages without the osgeo-
prefix are still a dependency to the new naming convention. Or am I mistaken?
Warning: You have unlinked kegs in your Cellar.
Leaving kegs unlinked can lead to build-trouble and cause brews that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
osgeo-sip
osgeo-pyqt
Warning: Some installed formulae are missing dependencies.
You should `brew install` the missing dependencies:
brew install libgeotiff libspatialite netcdf open-scene-graph proj
Run `brew missing` for more details.
~/ » brew missing 23:11:47
gdal: proj libgeotiff libspatialite netcdf
liblas: proj libgeotiff libspatialite netcdf
osgeo-gdal-python: proj libgeotiff libspatialite netcdf
osgeo-grass: proj libgeotiff netcdf libspatialite
osgeo-liblas: proj libgeotiff libspatialite netcdf
osgeo-orfeo: netcdf proj libgeotiff open-scene-graph libspatialite
osgeo-ossim: open-scene-graph proj libgeotiff libspatialite netcdf
osgeo-pyspatialite: proj libspatialite
osgeo-qgis-res: proj libgeotiff libspatialite netcdf
osgeo-saga-lts: proj netcdf libgeotiff libspatialite
osgeo-taudem: proj libgeotiff libspatialite netcdf
pcl: netcdf
pdal: proj libgeotiff libspatialite netcdf
vtk: netcdf
Linking the packages leads to error sins e.g sip
is present: should I overwrite?
And if I want to e.g reinstall osgeo-grass
:
~/ » brew reinstall osgeo-grass 23:18:27
osgeo-postgresql: You have other linked versions!
Unlink with brew unlink postgresql or remove with brew uninstall --ignore-dependencies postgresql
osgeo-gdal: You have other linked versions!
Unlink with brew unlink gdal or remove with brew uninstall --ignore-dependencies gdal
Error: Unsatisfied requirements failed this build.
@LeonardAukea
@fjperini Thanks. but something seems to be messy still. Looks like the packages without the
osgeo-
prefix are still a dependency to the new naming convention. Or am I mistaken?
there are tons of packages without the osgeo-*
prefix that are still a dependency. We just brought to the tap the most problematic and important ones.
Warning: You have unlinked kegs in your Cellar. Leaving kegs unlinked can lead to build-trouble and cause brews that depend on those kegs to fail to run properly once built. Run `brew link` on these: osgeo-sip osgeo-pyqt Warning: Some installed formulae are missing dependencies. You should `brew install` the missing dependencies: brew install libgeotiff libspatialite netcdf open-scene-graph proj Run `brew missing` for more details. ~/ » brew missing 23:11:47 gdal: proj libgeotiff libspatialite netcdf liblas: proj libgeotiff libspatialite netcdf osgeo-gdal-python: proj libgeotiff libspatialite netcdf osgeo-grass: proj libgeotiff netcdf libspatialite osgeo-liblas: proj libgeotiff libspatialite netcdf osgeo-orfeo: netcdf proj libgeotiff open-scene-graph libspatialite osgeo-ossim: open-scene-graph proj libgeotiff libspatialite netcdf osgeo-pyspatialite: proj libspatialite osgeo-qgis-res: proj libgeotiff libspatialite netcdf osgeo-saga-lts: proj netcdf libgeotiff libspatialite osgeo-taudem: proj libgeotiff libspatialite netcdf pcl: netcdf pdal: proj libgeotiff libspatialite netcdf vtk: netcdf
Linking the packages leads to error sins e.g
sip
is present: should I overwrite?
You can have both sip
s if you want, but our recommendation is to have just one... of course you can have linked just one of them and if you update osgeo-qgis
is going to install, link and so on and so forth the one from this tap.
If you want to have both sip
s you are probably going to need to juggle a little bit with them, in other words decide which one you one linked and/or over the other and cope with the consequences.
I'm doing the same with r
since I don't have the homebrew-core install on my computer. So each time I reinstall/update osgeo-qgis
I have to rename that keg so osgeo-qgis
install r
from core, and then delete that keg and replace with the one I like.
And if I want to e.g reinstall
osgeo-grass
:~/ » brew reinstall osgeo-grass 23:18:27 osgeo-postgresql: You have other linked versions! Unlink with brew unlink postgresql or remove with brew uninstall --ignore-dependencies postgresql osgeo-gdal: You have other linked versions! Unlink with brew unlink gdal or remove with brew uninstall --ignore-dependencies gdal Error: Unsatisfied requirements failed this build.
With this is the same... you still have the other versions installed...
If you are using those formulae just with QGIS the recommendation is to uninstall all the other and let osgeo-qgis
do it's business during the install.
The idea is to have a QGIS as complete as possible. For this reason so many dependencies are installed, ffmpeg corresponds to osgeo-grass.
@fjperini, a bit like @grischard I don't understand what are all these dependencies for ? I understand the idea but maybe that these formulae install a Qgis version which is too complete ? On my computer, qgis-ltr
and his dependencies use 11 Go of disk space! It's considerably more than the 2Go of data installed by official advanced installer for a base qgis-ltr
installation on windows computer.
Of course, it's a base installation without many add-ons but that's enough for me. For me, less is better. I would prefer can choose to install or not osgeo-saga-lts
, osgeo-grass
, osgeo-lastools
, osgeo-taudem
, r
, and surely more.
Maybe formulae should be base on the official installer with similar options for more simplicity ?
And if the advanced user want use a specific add-on, he can install it using an option or simply install "optional" package.
Ok! In the course of the day I will be doing some tests.
We will be working, some inconveniences with proj6 and gdal3 arose. Do not update at the moment.
@grischard is almost decided, we will use gdal 2.4.1 and proj 5.0.0. Just wait until I apply all the changes.
Keep in mind that for osgeo-proj
, osgeo-gdal
, osgeo-gdal-*
and osgeo-libgeotiff
you will have to uninstall and reinstall. And then continue with the other updates.
In case homebrew does not update these packages
While I really appreciate the ability to install QGIS from Homebrew and all the work that must have gone into this, I just want to add an enormous +1 to the request for cutting down the gigantic dependencies tree.
I understand that all the dependencies are required for building the QGIS Bottle with support for all the modules on the server, but is it really necessary to specify them as dependencies when installing the Formula on the user side? Even if QGIS was built with support for certain modules, it's still possible to run the build without most of the current dependencies (as I'm currently doing), and it would only throw up appropriate error messages once the users attempts to use a functionality that requires other libraries. You say that a large part of the community uses many of the tools, but I doubt that even a power user ever makes use of even half of those dependencies. Things like the full postgresql and mysql servers aren't actually required by anyone who doesn't know how to install them themselves (apart from the fact that the dependency should really only be some (client) header files and not the full server formula, no?).
I gave up full installation about half way through the dependencies, then just installed with --ignore-dependencies
and now have a seemingly fully functional QGIS running. If you want I can try to figure out where in the dependency tree I aborted the installs so that other users can also make use of a much smaller installation this way.
@kevinstadler we will create an option for those that decide a simpler installation, first we are solving other problems with proj and gdal.
Just wait for the process to complete.
Hurray to the option for a simpler installation with less dependencies. Thank you @fjperini for all your hard work!
Thanks so much! Just one more slightly unrelated question, is there a specific formula for installing QGIS with Python plugin support? For me the plugin menu just says "no Python support detected" even though I'm pretty sure I have all necessary dependencies, and I've also manually put a plugin in the profile/default/python/plugins
folder but it's not showing up. I'm using the osgeo-qgis (not LTR) formula right now.
@kevinstadler It should work. You could share the plugin that you put so I try it.
@fjperini thanks for your help! It's the profiletool plugin: https://plugins.qgis.org/plugins/profiletool/
I've put the extracted profiletool/
folder into what I think is the appropriate subfolder in the active QGIS profile:
/Users/kevin/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/
The python/
folder already existed but only had an expressions/
subdirectory, so I had to create the plugins/
one myself.
@kevinstadler After updating QGIS I will check this.
@kevinstadler I have checked this and the plugin works correctly.
cd /usr/local/Cellar/osgeo-qgis/3.6.3/QGIS.app/Contents/Resources/python/plugins/
git clone https://github.com/PANOimagen/profiletool
Here are some ideas we should keep in mind about the renaming of the formulae and if we should bring dependencies into the tap. This issue is highly related to #769.
Any opinion is highly welcomed. @fjperini and specially @miguelangelperg that seems that it's suffering from "name issues".
More formulae → more to maintain → more work → more difficulty to keep everything in shape.
gdal
,pdal
,postgis
... We have some of those formulae cloned in this tap with a different name so it doesn't shadowed the core formulae. What we should do?depends_on:
in any core or other tap formulae unless otherwise is specify is going to use core formulae. Probably they are going to be linked too first.keg only
so it doesn't interfere with other formulae that could use the core version, but this is something we need to decide.osgeo-<formula_name>
.<formula-name>@<version-number>
Thoughts?