Closed spittssj closed 3 years ago
carlocab
commented
3 years ago if
brew gist-logsdidn't work: ranbrew configandbrew doctorand included their output with your issue?
Where is the output of brew config and brew doctor? Please don't tick checkboxes without actually doing them.
SMillerDev
commented
3 years ago Could you fill out the issue template please? It's pretty essential in debugging issues. This issue can be reopened once we have all the information needed to start debugging.
spittssj
commented
3 years ago Brew doctor reports "Your system is ready to brew."
Here is brew config: HOMEBREW_VERSION: 2.7.5 ORIGIN: https://github.com/Homebrew/brew HEAD: bca4804a9e48de5319383d3eddadaa7f054c77da Last commit: 7 days ago Core tap ORIGIN: https://github.com/Homebrew/homebrew-core Core tap HEAD: ab61019e86e72aced43da0e56da457099e580992 Core tap last commit: 11 hours ago Core tap branch: master HOMEBREW_PREFIX: /usr/local HOMEBREW_CASK_OPTS: [] HOMEBREW_MAKE_JOBS: 8 Homebrew Ruby: 2.6.3 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby CPU: octa-core 64-bit icelake Clang: 12.0 build 1200 Git: 2.30.0 => /usr/local/bin/git Curl: 7.64.1 => /usr/bin/curl macOS: 11.1-x86_64 CLT: 12.3.0.0.1.1607026830 Xcode: 12.3
"Could you fill out the issue template please?" I thought that was what I was doing with the headings I used above. What other information do you need?
"Please don't tick checkboxes without actually doing them." Please be kind. This is the first time I've reported an issue.
matthieutirelli-pro
commented
3 years ago I am currently running into the same issue as described by @spittssj. Here the gist with all the required details you ask for :
brew gist-logs libspatialite
https://gist.github.com/91c81868a1a3ef5e6ebf8658e02d1283And the error :
>>> import geopandas
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/matthieutirelli/PycharmProjects/venv-pycharm-neotac-intercept/lib/python3.9/site-packages/geopandas/__init__.py", line 7, in <module>
from geopandas.io.file import _read_file as read_file # noqa
File "/Users/matthieutirelli/PycharmProjects/venv-pycharm-neotac-intercept/lib/python3.9/site-packages/geopandas/io/file.py", line 7, in <module>
import fiona
File "/Users/matthieutirelli/PycharmProjects/venv-pycharm-neotac-intercept/lib/python3.9/site-packages/fiona/__init__.py", line 86, in <module>
from fiona.collection import BytesCollection, Collection
File "/Users/matthieutirelli/PycharmProjects/venv-pycharm-neotac-intercept/lib/python3.9/site-packages/fiona/collection.py", line 11, in <module>
from fiona.ogrext import Iterator, ItemsIterator, KeysIterator
ImportError: dlopen(/Users/matthieutirelli/PycharmProjects/venv-pycharm-neotac-intercept/lib/python3.9/site-packages/fiona/ogrext.cpython-39-darwin.so, 2): Symbol not found: _GEOSArea
Referenced from: /usr/local/opt/libspatialite/lib/libspatialite.7.dylib
Expected in: flat namespace
in /usr/local/opt/libspatialite/lib/libspatialite.7.dylib
>>> Ah, I tried
brew gist-logs gdaland got
No logs.I didn't think of doing it as well for libspatialite until now. Here is my brew gist-logs output as well, also for libspatialite.
https://gist.github.com/29353099c08d93dda32868cab942e1a6
I think the otools output from the original message could be relevant too. One thing that makes me nervous is that MacOS X also has its own sqlite installed in /usr/lib/sqlite3.
If there is any other info that would be helpful, let me know.
fxcoudert
commented
3 years ago It looks like a geopandas build issue. Have you reported it to them? They might have a better idea of where this comes from.
spittssj
commented
3 years ago I opened an issue with geopandas too and have yet to hear a response. I agree that something is not playing nice at the C level with the shared libraries. I have looked around to no avail for other installs of gdal (often a culprit). I'm afraid they will say, "we don't support homebrew + pip, use conda" but evidently conda does not place nice with homebrew.
Since my postgis install is managed as well through brew, I'd like to keep the whole GIS toolchain in one package manager lest it get even worse.
I am a PhD student at the University of Minnesota in the US. This is a grant funded project. We will happily make a donation to Homebrew if someone can help me debug this.
alebcay
commented
3 years ago The missing symbol error looks a bit like https://github.com/Homebrew/homebrew-core/issues/5161, not sure if related. MacPorts has also had a similar issue (seems unresolved though) with geopandas as well: https://trac.macports.org/ticket/58718.
fxcoudert
commented
3 years ago After reading the old issue and the macports ticket, what I think is happening is a conflict with another version of libspatialite. It looks like installing geopandas linked against another version of the library (possibly found somewhere else on your system) and if Homebrew's version gets loaded, it then can find the same symbols.
spittssj
commented
3 years ago Dear all,
@alebcay yes, if you read the original issue I mentioned #5161. I also saw the MacPorts issue. The thing is -- no one has tried to debug it. I included the otools -L output above for that reason.
@fxcoudert I thought of that too. That's why I included the gist of the build from source for libspatialite. The tricky thing is that geopandas doesn't link with any C libraries directly and
sudo find / -name *spatialite* -printreports no other copies of libspatiallite on my system.
I got a detailed reply from a geopandas developer. They admit that there are some subtle bugs involving the C layer of the import order.
https://github.com/geopandas/geopandas/issues/1786
As I suspected, the answer is "install geopandas from conda-forge." So that means I would have four python installs on my system: 1) Apple's provided python2.7 2) XCode SDK's python3.8 3) Homebrew's python3.9 (connected to other Homebrew packages like postgresql and postgis) 4) Anaconda and whatever it brings
I would like to avoid this situation: https://xkcd.com/1987/
What is the best practice that other Python GIS developers use? Maybe it is better to keep a "system python" with homebrew and a "development python" with Anaconda with my projects. I'm still learning about all of these things.
I appreciate all of your help.
Stephen
fxcoudert
commented
3 years ago OK, with
geopandas doesn't link with any C libraries directly
and
They admit that there are some subtle bugs involving the C layer of the import order
it means it's out of Homebrew's scope for sure. This is a bug in our these different python packages use ctype to load/import the libraries.
It seems you have a workaround (changing the import order). Fur a longer-term solution, geopandas should fix their issue. While this may be an interesting discussion to have on their issue tracker or discussion forum, there is nothing we can do here, so I'm closing.
fxcoudert
commented
3 years ago Regarding this:
I am not familiar myself with how Howebrew works (I also don't have a Mac). It might also that the packages are not fully ready for the latest version of MacOSX
Given that the packages built fine on our CI, and built fine from source for you, I don't think that's the problem.
Bug report
Please note we will close your issue without comment if you delete, do not read or do not fill out the issue checklist below and provide ALL the requested information. If you repeatedly fail to use the issue template, we will block you from ever submitting issues to Homebrew again.
brew updateand can still reproduce the problem?brew doctor, fixed all issues and can still reproduce the problem?brew gist-logs <formula>(where<formula>is the name of the formula that failed) and included the output link?brew gist-logsdidn't work: ranbrew configandbrew doctorand included their output with your issue?What you were trying to do (and why)
I upgraded to MacOS Big Sur 11.1 recently and updated my Python GIS set of packages to continue working on a project that uses GeoDjango + GeoPandas + Postgis. Postgis works fine, but I am getting a strange error when trying to use geopandas below.
What happened (include command output)
With a fresh install of python@3.9 port, if I don't import fiona before geopandas, then I get a strange linking error below.
Importing fiona before geopandas works perfectly
On the other hand,
Importing geopandas by itself breaks
(env) stephenpitts@Ellacuria env % otool -L /usr/local/opt/libspatialite/lib/libspatialite.7.dylib /usr/local/opt/libspatialite/lib/libspatialite.7.dylib: /usr/local/opt/libspatialite/lib/libspatialite.7.dylib (compatibility version 9.0.0, current version 9.1.0) /usr/local/opt/libtiff/lib/libtiff.5.dylib (compatibility version 12.0.0, current version 12.0.0) /usr/lib/libcurl.4.dylib (compatibility version 7.0.0, current version 9.0.0) /usr/local/opt/libxml2/lib/libxml2.2.dylib (compatibility version 12.0.0, current version 12.10.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1) /usr/local/opt/minizip/lib/libminizip.1.dylib (compatibility version 2.0.0, current version 2.0.0) /usr/local/opt/librttopo/lib/librttopo.1.dylib (compatibility version 3.0.0, current version 3.0.0) /usr/local/opt/freexl/lib/libfreexl.1.dylib (compatibility version 3.0.0, current version 3.0.0) /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0) /usr/local/opt/proj/lib/libproj.19.dylib (compatibility version 22.0.0, current version 22.1.0) /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11) /usr/local/opt/sqlite/lib/libsqlite3.0.dylib (compatibility version 9.0.0, current version 9.6.0) /usr/local/opt/geos/lib/libgeos_c.1.dylib (compatibility version 18.0.0, current version 18.2.0)What you expected to happen
I expected geopandas to import fine above.
Step-by-step reproduction instructions (by running
brew installcommands)I have rebuilt the entire set of packages from source to no avail.
brew install -s python brew install -s gdal python -m venv env cd env; source env/bin/activate pip install --no-binary :all: geopandas
The setup worked under the previous release of MacOS X and homebrew, and my collaborator uses MacOS X and homebrew (but is afraid to upgrade after I shared my experience), so I am reluctant to switch to conda, as some forum posts I have seen have suggested.
I have read about subtle issues with import ordering and ctypes on various other posts about problems with Geopandas. I could attach "python -v" output from the above if someone were interested. I've used Python on and off for 10 years and I'm frustrated that I don't even know how to debug this one.
I'm starting here because I found issue #5161 from four years ago with the same problem and I figure the out of the box Python GIS setup should work on Homebrew (hence I have probably set up something wrong).
Thanks for any help.
Stephen