matplotlib / basemap

Plot on map projections (with coastlines and political boundaries) using matplotlib
MIT License
780 stars 392 forks source link

pip error on Macbook pro (m1 pro) #539

Closed CaffreyR closed 2 years ago

CaffreyR commented 2 years ago

Hi! I am working on Macbook pro with basemap, but I met some issues. When I try to pip install basemap, there is something wrong with this. But I have already brew install geos, what's wrong? Thanks! image

molinav commented 2 years ago

Hi @CaffreyR! Did you set the GEOS_DIR environment variable? Since your traceback says that the file "geos_c.h" is not found, I would assume that the _geoslib module cannot be compiled because of not finding the GEOS headers in your system.

ronaldbradford commented 2 years ago

I'm getting the same problem, and I'm setting GEOS_DIR. Brew installs geos=3.10.2, this comment in basemap suggests installing an older version https://github.com/matplotlib/basemap/issues/533

molinav commented 2 years ago

Hi @ronaldbradford, thanks for the feedback. Could you confirm me if you can find a "geos_c.h" file in your system and the value that you are assigning to GEOS_DIR?

ronaldbradford commented 2 years ago

I was using GEOS_DIR=/usr/local/Cellar/geos/3.10.2

I've uninstalled and building form source right now to see if I can confirm it fixes my issuel

ronaldbradford commented 2 years ago

I have installed geos 3.8.2 from source https://libgeos.org/usage/download/

$ ls -l $GEOS_DIR/include/geos_c.h
-rw-r--r--  1 rbradfor  staff  97001 Apr  7 17:26 /usr/local/include/geos_c.h
$ echo $GEOS_DIR
/usr/local

$ pip3 install basemap
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621
Collecting basemap
  Using cached basemap-1.3.2.zip (155 kB)
  ...
  Building wheels for collected packages: basemap
  Building wheel for basemap (pyproject.toml) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/local/opt/python@3.9/bin/python3.9 /usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /var/folders/hp/l11pqpjn721d4cjpw1qjzl_c0000gn/T/tmpjrhbdnvf
       cwd: /private/var/folders/hp/l11pqpjn721d4cjpw1qjzl_c0000gn/T/pip-install-0vpt9rzd/basemap_ce944475e96546c5846addd29810dfff
  Complete output (32 lines):
  setup.py:52: RuntimeWarning: Cannot find GEOS library and/or headers in standard locations ('/usr/local'). Please install the corresponding packages using your software management system or set the environment variable GEOS_DIR to point to the location where GEOS is installed (for example, if 'geos_c.h' is in '/usr/local/include' and 'libgeos_c' is in '/usr/local/lib', then you need to set GEOS_DIR to '/usr/local'
    warnings.warn(" ".join([
  running bdist_wheel
  running build
  running build_py
  ...
ronaldbradford commented 2 years ago

FWIW 'm also getting the same error with

$ pip3 install basemap==1.3.1
$ pip3 install basemap==1.3.0
molinav commented 2 years ago

From your traceback I would infer that there is no problem with finding the header file but with finding the library itself.

The problematic line could be this one in the setup file: https://github.com/matplotlib/basemap/blob/f9c9dabd2e8e41b7573c5a8b1beaf7182ea87173/packages/basemap/setup.py#L45

The setup file tries to find the library assuming that the library filename follows the pattern "libgeos_c*.so*" and is placed inside $GEOS_DIR/lib, $GEOS_DIR/lib/x86_64-linux-gnu or $GEOS_DIR/lib64. Is this correct for MacOS? If not, then the setup file would need an update.

ronaldbradford commented 2 years ago

I have installed geos 3.9.2 now from source https://libgeos.org/usage/download/ I Installed basemap from source via https://github.com/matplotlib/basemap instructions Debugging setup.py is beyond my knowledge level. I know it's late there, please let me know what I can try.

$  ls -l $GEOS_DIR/include/geos_c.h
-rw-r--r--  1 rbradfor  staff  79397 Apr  7 17:53 /usr/local/include/geos_c.h
$ python -m pip install .
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621
Processing /Users/rbradfor/tmp/basemap/packages/basemap
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy<1.23,>=1.21 in /usr/local/lib/python3.9/site-packages (from basemap==1.3.2+dev) (1.22.2)
Requirement already satisfied: basemap-data<1.4,>=1.3.2 in /usr/local/lib/python3.9/site-packages (from basemap==1.3.2+dev) (1.3.2)
Requirement already satisfied: matplotlib<3.6,>=1.5 in /usr/local/lib/python3.9/site-packages (from basemap==1.3.2+dev) (3.5.1)
Collecting pyshp<2.2,>=1.2
  Using cached pyshp-2.1.3-py3-none-any.whl
Requirement already satisfied: pyproj<3.4.0,>=1.9.3 in /usr/local/lib/python3.9/site-packages (from basemap==1.3.2+dev) (3.3.0)
Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.9/site-packages (from matplotlib<3.6,>=1.5->basemap==1.3.2+dev) (9.0.1)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.9/site-packages (from matplotlib<3.6,>=1.5->basemap==1.3.2+dev) (1.3.2)
Requirement already satisfied: pyparsing>=2.2.1 in /usr/local/lib/python3.9/site-packages (from matplotlib<3.6,>=1.5->basemap==1.3.2+dev) (3.0.7)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.9/site-packages (from matplotlib<3.6,>=1.5->basemap==1.3.2+dev) (21.3)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.9/site-packages (from matplotlib<3.6,>=1.5->basemap==1.3.2+dev) (4.29.1)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.9/site-packages (from matplotlib<3.6,>=1.5->basemap==1.3.2+dev) (0.11.0)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.9/site-packages (from matplotlib<3.6,>=1.5->basemap==1.3.2+dev) (2.8.2)
Requirement already satisfied: certifi in /usr/local/lib/python3.9/site-packages (from pyproj<3.4.0,>=1.9.3->basemap==1.3.2+dev) (2021.10.8)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib<3.6,>=1.5->basemap==1.3.2+dev) (1.16.0)
Building wheels for collected packages: basemap
  Building wheel for basemap (pyproject.toml) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/local/opt/python@3.9/bin/python3.9 /usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /var/folders/hp/l11pqpjn721d4cjpw1qjzl_c0000gn/T/tmp8iq42jtw
       cwd: /Users/rbradfor/tmp/basemap/packages/basemap
  Complete output (19 lines):
  setup.py:52: RuntimeWarning: Cannot find GEOS library and/or headers in standard locations ('/usr/local'). Please install the corresponding packages using your software management system or set the environment variable GEOS_DIR to point to the location where GEOS is installed (for example, if 'geos_c.h' is in '/usr/local/include' and 'libgeos_c' is in '/usr/local/lib', then you need to set GEOS_DIR to '/usr/local'
    warnings.warn(" ".join([
  running bdist_wheel
  running build
  running build_py
  running build_ext
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -I/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/numpy/core/include -I/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c src/_geoslib.c -o build/temp.macosx-10.15-x86_64-3.9/src/_geoslib.o
  In file included from src/_geoslib.c:685:
  In file included from /usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/numpy/core/include/numpy/arrayobject.h:5:
  In file included from /usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/numpy/core/include/numpy/ndarrayobject.h:12:
  In file included from /usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/numpy/core/include/numpy/ndarraytypes.h:1960:
  /usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: "Using deprecated NumPy API, disable it with "          "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
  #warning "Using deprecated NumPy API, disable it with " \
   ^
  src/_geoslib.c:686:10: fatal error: 'geos_c.h' file not found
  #include "geos_c.h"
           ^~~~~~~~~~
  1 warning and 1 error generated.
  error: command '/usr/bin/clang' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for basemap
Failed to build basemap
ERROR: Could not build wheels for basemap, which is required to install pyproject.toml-based projects
WARNING: You are using pip version 21.3.1; however, version 22.0.4 is available.
You should consider upgrading via the '/usr/local/opt/python@3.9/bin/python3.9 -m pip install --upgrade pip' command.
ronaldbradford commented 2 years ago
$ ls -l $GEOS_DIR/lib/libgeos_c*
-rwxr-xr-x  1 rbradfor  staff   226536 Apr  7 17:35 /usr/local/lib/libgeos_c.1.13.4.dylib
-rwxr-xr-x  1 rbradfor  staff   233112 Apr  7 17:49 /usr/local/lib/libgeos_c.1.14.3.dylib
-rwxr-xr-x  1 rbradfor  staff  1407648 Apr  7 17:58 /usr/local/lib/libgeos_c.1.9.1.dylib
lrwxr-xr-x  1 rbradfor  staff       21 Apr  7 17:58 /usr/local/lib/libgeos_c.1.dylib -> libgeos_c.1.9.1.dylib
lrwxr-xr-x  1 rbradfor  staff       17 Apr  7 17:35 /usr/local/lib/libgeos_c.dylib -> libgeos_c.1.dylib
$ uname -a
Darwin atlantis.lan 19.6.0 Darwin Kernel Version 19.6.0: Sun Nov 14 19:58:51 PST 2021; root:xnu-6153.141.50~1/RELEASE_X86_64 x86_64

Let me interactively look at ~L45 and see.

ronaldbradford commented 2 years ago
>>> import os
>>> os.name
'posix'
>>> libfiles = []
>>> libdirs = ["bin", "lib", "lib/x86_64-linux-gnu", "lib64"]
>>> libext = "dll" if os.name == "nt" else "so"
>>> libcode = "{0}geos_c".format("" if os.name == "nt" else "lib")
>>> libname = "{0}*.{1}*".format(libcode, libext)
>>> for libdir in libdirs:
...   libfiles.extend(glob.glob(os.path.join(prefix, libdir, libname)))
>>> print(libfiles)
[]
>>> print("{},{},{}".format(libext,libcode,libname))
so,libgeos_c,libgeos_c*.so*

So yes, there is no *.so file on MacOS

ronaldbradford commented 2 years ago

https://github.com/matplotlib/basemap/commit/41214409b49461cee8b65dd25fb33ed8ebd9963b

molinav commented 2 years ago

This traceback shows much more information. Something is going wrong when setting the compilation flags internally, because the clang call is missing a -I$GEOS_DIR/include (with $GEOS_DIR expanded to your value). This is supposed to be done by the setup.py but it is clear that it is not doing its job in MacOS.

Can you check if setting the environment variable CFLAGS="-I$GEOS_DIR/include" in advance lets you go further with the installation?

molinav commented 2 years ago

https://github.com/matplotlib/basemap/commit/41214409b49461cee8b65dd25fb33ed8ebd9963b

You got it. This commit done by me is causing at least one of the problems. Before, only the presence of the header file was checked. I wanted to ensure that the library was actually there too, but only considered the extensions for Windows (.dll) and for manylinux (.so).

I will need to update this. But still I am confused, because in your other traceback the first check seems to pass but later the compilation of _geoslib fails due to the missing clang header argument.

What is clear is that I should create a GitHub Action to build MacOS wheels, because with them this error would have appeared fast.

It is a bit late for me now but I will try to take a look to this either tomorrow afternoon or this weekend.

CaffreyR commented 2 years ago

Hi @CaffreyR! Did you set the GEOS_DIR environment variable? Since your traceback says that the file "geos_c.h" is not found, I would assume that the _geoslib module cannot be compiled because of not finding the GEOS headers in your system.

I'm getting the same problem, and I'm setting GEOS_DIR. Brew installs geos=3.10.2, this comment in basemap suggests installing an older version #533

Yep, I have set the dir in .bash_profile too. As you can see above. @molinav

image

And does an older version work? @ronaldbradford

molinav commented 2 years ago

@CaffreyR You need to define GEOS_DIR as an independent environment variable instead of adding the GEOS directory to PATH, like so:

export GEOS_DIR="/opt/homebrew/Cellar/geos/3.10.2"

Still, this won't be enough because the setup file needs a correction as shown by @ronaldbradford above. Here the GEOS version does not play a role: the recommendation about the version limit for GEOS is due to a problem only when drawing the Antarctic coast.

molinav commented 2 years ago

@ronaldbradford I have just pushed some commits to the bugfix-539 branch. There, I reimplement how the GEOS library is searched so that it also considers .dylib extensions on MacOS. Could you give a try and install from this branch to see if your problem is solved on your system?

molinav commented 2 years ago

@CaffreyR May I ask you to try my attempt of fixing your problem? You need to do:

export GEOS_DIR="/opt/homebrew/Cellar/geos/3.10.2"
git clone https://github.com/matplotlib/basemap.git
cd basemap/packages/basemap
git checkout bugfix-539
python -m pip install .
CaffreyR commented 2 years ago

@CaffreyR May I ask you to try my attempt of fixing your problem? You need to do:

export GEOS_DIR="/opt/homebrew/Cellar/geos/3.10.2"
git clone https://github.com/matplotlib/basemap.git
cd basemap/packages/basemap
git checkout bugfix-539
python -m pip install .

OK!But there's something wrong with the git , maybe for network in my region, let me try it again later. @molinav

molinav commented 2 years ago

@CaffreyR Be sure not to use --depth 1 when cloning, because then the history is not downloaded and you cannot change to the bugfix branch. I wrote it wrong and edited my message, but if you copied the snippet from an email notification it will still be there.

CaffreyR commented 2 years ago

@CaffreyR Be sure not to use --depth 1 when cloning, because then the history is not downloaded and you cannot change to the bugfix branch. I wrote it wrong and edited my message, but if you copied the snippet from an email notification it will still be there.

see there is continuously disconnection, whether I use my VPN. Is there any chance that git broken? @molinav

image
molinav commented 2 years ago

@CaffreyR It looks more like a network issue then. Maybe you can try a bit later in case the connection gets better.

ronaldbradford commented 2 years ago

@molinav +1

...
Installing collected packages: pyshp, basemap

$ pip list | grep basemap
basemap                                  1.3.2+dev
basemap-data                             1.3.2
basemap-data-hires                       1.3.2
Screen Shot 2022-04-15 at 10 10 51 AM
molinav commented 2 years ago

@ronaldbradford Thanks! Great to see that it works. I will prepare a patch release 1.3.3 soon so that the source distributions from PyPI also work for MacOS.

CaffreyR commented 2 years ago

@molinav Finnally!!!! I downloaded the git! It really worked! Thanks so much!

image
molinav commented 2 years ago

@CaffreyR Nice to hear! Thanks!

isimo00 commented 2 years ago

I'll leave an update on how to solve this since even bug #539 of basemap has been fixed but I still had this issue even after. You want to make sure you first run brew install geos

Then, @molinav 's lines might need some modifications. Check where geos is being stored, it might not be "/opt/homebrew/Cellar/geos/3.10.2". This will be printed when you install geos using brew. For me, it was rather

export GEOS_DIR = "/usr/local"

Then,

cd basemap/packages/basemap

And again, this will be different since the bugfix-539 branch has already been merged and deleted;

git checkout develop

And finally

python -m pip install .

Summary:

brew install geos
export GEOS_DIR = "/usr/local"
cd basemap/packages/basemap
git checkout develop
python -m pip install .
molinav commented 2 years ago

@isimo00 Thanks for the feedback. I recently created a hotfix version 1.3.3 with the corrections for Mac and pushed it to PyPI: https://pypi.org/project/basemap/1.3.3/

Although there are no precompiled wheels for Mac, now you can install basemap from the PyPI source distribution, so in principle it would be enough with doing the following (assuming that you have a C compiler):

brew install geos
export GEOS_DIR="/usr/local"
python -m pip install basemap

The good thing of this approach is that you do not need to deal with the specific commands related to using a repository clone.

isimo00 commented 2 years ago

@molinav Oh great! Thanks for the answer and work, hopefully this will make it easier for future users.

jucor commented 2 years ago

Thanks a lot @molinav ! Came here precisely to find this ❤️ I can say it works here too, with the slight adjustment of export GEOS_DIR="/opt/homebrew".

Any chance to get a wheel at some point to conda-forge, please?

molinav commented 2 years ago

@jucor Actually there was some work recently in the corresponding feedstock, and we managed to make the pipelines pass again with the new basemap 1.3.x series. I am waiting for some feedback from the other people involved in the PR just to be sure that everything also looks ok for them... And then we will have basemap back in conda-forge. ;-)

ihaseeburrehman commented 1 year ago

hi for me its not working here are the steps which did 1-download the basemap 2- export GEOS_DIR=/opt/homebrew/Cellar/geos/3.11.1/ 3- cd basemap/packages/basemap 4-python -m pip install basemap

but failed to build wheel for basemap. image

molinav commented 1 year ago

Hi @ihaseeburrehman. I am not an expert on macOS and I have no macOS machine to test, but this seems to be some issue related to the build toolchain rather than basemap itself, see for example this issue:

https://github.com/llvm/llvm-project/issues/55836

Since you were able to install basemap successfully in your homebrewed Python based on https://github.com/matplotlib/basemap/issues/547#issuecomment-1352890489, looking at your environment variables could help you find a solution (to ensure that Anaconda and your SDK do not conflict).

xpivan commented 1 year ago

@CaffreyR May I ask you to try my attempt of fixing your problem? You need to do:

export GEOS_DIR="/opt/homebrew/Cellar/geos/3.10.2"
git clone https://github.com/matplotlib/basemap.git
cd basemap/packages/basemap
git checkout bugfix-539
python -m pip install .

Hello here :) While running git checkout bugfix-539 I have the following error: error: pathspec 'bugfix-539' did not match any file(s) known to git

molinav commented 1 year ago

Hi @xpivan! The bugfix-539 branch was a temporary branch to solve this issue, but it was removed as soon as it was merged back to the main branches. You can try to install from any stable tag, e.g. for version 1.3.8 it would be as below:

python -m pip install https://github.com/matplotlib/basemap/archive/v1.3.8.tar.gz#subdirectory=packages/basemap

Please note that until I release basemap version 1.3.9, you should ensure that you have numpy < 1.24, otherwise you might be hit by the bug reported recently in #589.

xpivan commented 1 year ago

Hi @molinav

Thanks for your answer. I changed my numpy version to 1.23.0 My python version is 3.8.10 I'm also using Mac M1 Chip

Using the command

python -m pip install https://github.com/matplotlib/basemap/archive/v1.3.8.tar.gz#subdirectory=packages/basemap

returns the same problem:

:58: RuntimeWarning: Cannot find GEOS library and/or headers in standard locations ('/opt/homebrew/Cellar/geos/3.10.2'). Please install the corresponding packages using your software management system or set the environment variable GEOS_DIR to point to the location where GEOS is installed (for example, if 'geos_c.h' is in '/usr/local/include' and 'libgeos_c' is in '/usr/local/lib', then you need to set GEOS_DIR to '/usr/local' /private/var/folders/s1/t3dxzy3s3fqccsfjkhggz3fr0000gn/T/pip-build-env-u1zn9dhn/overlay/lib/python3.8/site-packages/setuptools/dist.py:700: SetuptoolsDeprecationWarning: The namespace_packages parameter is deprecated. !! ******************************************************************************** Please replace its usage with implicit namespaces (PEP 420). See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages for details. ******************************************************************************** !! ep.load()(self, ep.name, value) error: command '/usr/bin/gcc' failed with exit code 1 [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for basemap Failed to build basemap ERROR: Could not build wheels for basemap, which is required to install pyproject.toml-based projects [notice] A new release of pip is available: 23.1.2 -> 23.2.1 [notice] To update, run: pip install --upgrade pip
molinav commented 1 year ago

@xpivan Do you have GEOS on your system? When building basemap from source, you need GEOS available on your system (e.g. installed with homebrew). Once GEOS is installed, you need to define the environment variable GEOS_DIR with the location where GEOS was installed.

From the traceback, "/opt/homebrew/Cellar/geos/3.10.2" does not look like a valid GEOS installation root, so my first guess would be that you do not have the GEOS library in this location.

xpivan commented 1 year ago

@molinav GEOS was properly installed but I realized reading your message that my export was pointing to the wrong version. I changed the export from export GEOS_DIR="/opt/homebrew/Cellar/geos/3.10.2"

to

export GEOS_DIR="/opt/homebrew/Cellar/geos/3.12.0"

It works like a charm.

Thank you for taking time and helping solving for my issue. I really appreciated it.

Cheers

Isaac0880 commented 1 year ago

Hello, I have been trying to run code originally written for Python 2, but I have encountered some issues when attempting to run it

Traceback (most recent call last): File "PyVelest.py", line 27, in from mpl_toolkits.basemap import Basemap File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/mpl_toolkits/basemap/init.py", line 53, in import _geoslib ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_geoslib.so, 0x0002): symbol not found in flat namespace '_GEOSArea'

I have done everything that others have suggested in this issue, but I don't have any idea what's going on. I use a Mac M2, and I don't know if there is any problem with Macs.

ofloveandhate commented 12 months ago

When I try to install basemap via pip3 on my Mac, I wish that instead of, or in addition to, failing to find the header file when I don't have geos installed, it would tell me to install geos.

molinav commented 12 months ago

Hi @ofloveandhate! The issue here is that with a pip installation you cannot recommend installing GEOS with pip, the GEOS installation is somehow a "manual" step to do before the pip install basemap call. We can still, of course, reformulate the error message to make it a bit more descriptive (e.g. common ways to install GEOS, either from homebrew, from conda, from GNU/Linux package manager, etc.).

Which is your current environment on MacOS (system version, Python version)? I am currently experimenting to create MacOS wheels with a local Mac mini (making GEOS be already bundled in the wheel), so it would be great if somebody can try them succesfully too before they become public. They seem to work in my computer, but I am not so experienced with Mac devices.

ofloveandhate commented 12 months ago

@molinav I'm on Python 3.11 provided by homebrew, on a 2020 Mac Mini with an M1 chip running MacOS 13.6 (I cannot change the MacOS version, it's managed by my university).

I guess I was mostly asking not for pip to provide geos, but to tell me that there's an expectation that I would have geos installed separately as a pre-requisite step. (Especially since I teach, and I was helping a student install things for a project, having never installed basemap before I didn't understand that they/I needed to install geos separately).

Additional context for why I think an improved warning / message would help: I had been previously conditioned to suspect a missing header problem as a package bug, due to a few similar bugs I encountered in other Python packages.

molinav commented 12 months ago

@ofloveandhate Thanks for the feedback, I will try to update the warning message in the following days so that the information for the end users is more clear. If it is ok for you, I will ask you for feedback once I prepare the new message.

In the meantime, may I ask you to try the following wheel? I built it for Python 3.11 on macOS Sonoma (14.1.2), but using an older SDK (11.3), so it should also work with previous macOS versions. Please replace first the extension ".zip" with ".whl", since GitHub does not allow to attach wheel files in issue comments: basemap-1.4.0.dev0-cp311-cp311-macosx_11_0_arm64.zip

After renaming the file to use ".whl" instead of ".zip", it should be possible to install basemap by typing the usual pip call (optionally with --ignore-installed in case basemap is already on your environment):

python -m pip install basemap-1.4.0.dev0-cp311-cp311-macosx_11_0_arm64.whl

This basemap wheel already bundles GEOS 3.6.5, so once installed it should work out of the box. It would be very useful if you can check that it works, because then I would be a bit more confident to upload these macOS wheels (built locally) to PyPI in the short future. I tried to build them with an even older SDK (11.0), but without success due to some missing C symbols, so it seems SDK 11.3 is the lowest version I can use to build GEOS 3.6.5.

molinav commented 11 months ago

@ofloveandhate May I ask if you found some time to try the wheel?

ofloveandhate commented 11 months ago

@molinav It was successful out of the box!


These are the homebrew packages I had installed: (I removed geos before testing your wheel)

brew list
==> Formulae
aom         freetype        ipython         libtiff         node            python-setuptools
autoconf        fribidi         isl         libtool         numpy           python@3.10
autoconf@2.69       gcc         jasper          libuv           openblas        python@3.11
automake        gd          jbig2dec        libvmaf         openexr         python@3.12
bison           gdbm            jpeg-turbo      libx11          openjdk         pyyaml
black           gdk-pixbuf      jpeg-xl         libxau          openjpeg        readline
boost           gettext         jupyterlab      libxcb          openssl@1.1     ruff
boost-python3       ghostscript     libavif         libxdmcp        openssl@3       shared-mime-info
brotli          giflib          libde265        libxext         pandoc          six
c-ares          glib            libevent        libxrender      pango           sqlite
ca-certificates     gmp         libheif         libyaml         pcre2           webp
cairo           graphite2       libidn          little-cms2     pixman          x265
cffi            graphviz        liblqr          lz4         plantuml        xorgproto
cmake           gts         libmpc          lzo         pycparser       xz
coreutils       harfbuzz        libnghttp2      m4          pydocstyle      zeromq
cython          highway         libomp          mpdecimal       pygments        zstd
eigen           hwloc           libpng          mpfr            python-certifi
eigenpy         icu4c           libraw          mpich           python-lsp-server
flex            imagemagick     librsvg         mypy            python-packaging
fontconfig      imath           libsodium       netpbm          python-psutil

and here's the output from pip as I installed:

python3 -m pip install basemap-1.4.0.dev0-cp311-cp311-macosx_11_0_arm64.whl
Processing ./basemap-1.4.0.dev0-cp311-cp311-macosx_11_0_arm64.whl
Requirement already satisfied: basemap-data<1.4,>=1.3.2 in /opt/homebrew/lib/python3.11/site-packages (from basemap==1.4.0.dev0) (1.3.2)
Requirement already satisfied: pyshp<2.4,>=1.2 in /opt/homebrew/lib/python3.11/site-packages (from basemap==1.4.0.dev0) (2.3.1)
Requirement already satisfied: matplotlib<3.8,>=1.5 in /opt/homebrew/lib/python3.11/site-packages (from basemap==1.4.0.dev0) (3.7.3)
Requirement already satisfied: pyproj<3.7.0,>=1.9.3 in /opt/homebrew/lib/python3.11/site-packages (from basemap==1.4.0.dev0) (3.5.0)
Requirement already satisfied: packaging<24.0,>=16.0 in /opt/homebrew/lib/python3.11/site-packages (from basemap==1.4.0.dev0) (23.0)
Requirement already satisfied: numpy<1.26,>=1.21 in /opt/homebrew/lib/python3.11/site-packages (from basemap==1.4.0.dev0) (1.24.2)
Requirement already satisfied: contourpy>=1.0.1 in /opt/homebrew/lib/python3.11/site-packages (from matplotlib<3.8,>=1.5->basemap==1.4.0.dev0) (1.0.7)
Requirement already satisfied: cycler>=0.10 in /opt/homebrew/lib/python3.11/site-packages (from matplotlib<3.8,>=1.5->basemap==1.4.0.dev0) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /opt/homebrew/lib/python3.11/site-packages (from matplotlib<3.8,>=1.5->basemap==1.4.0.dev0) (4.38.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/homebrew/lib/python3.11/site-packages (from matplotlib<3.8,>=1.5->basemap==1.4.0.dev0) (1.4.4)
Requirement already satisfied: pillow>=6.2.0 in /opt/homebrew/lib/python3.11/site-packages (from matplotlib<3.8,>=1.5->basemap==1.4.0.dev0) (9.4.0)
Requirement already satisfied: pyparsing>=2.3.1 in /opt/homebrew/lib/python3.11/site-packages (from matplotlib<3.8,>=1.5->basemap==1.4.0.dev0) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7 in /opt/homebrew/lib/python3.11/site-packages (from matplotlib<3.8,>=1.5->basemap==1.4.0.dev0) (2.8.2)
Requirement already satisfied: certifi in /opt/homebrew/lib/python3.11/site-packages (from pyproj<3.7.0,>=1.9.3->basemap==1.4.0.dev0) (2022.12.7)
Requirement already satisfied: six>=1.5 in /opt/homebrew/lib/python3.11/site-packages (from python-dateutil>=2.7->matplotlib<3.8,>=1.5->basemap==1.4.0.dev0) (1.16.0)
Installing collected packages: basemap
Successfully installed basemap-1.4.0.dev0
ofloveandhate commented 11 months ago

(thanks for both your patience, and your follow-through!)

molinav commented 11 months ago

So finally we have the first basemap packages in conda-forge for MacOS ARM64 (since version 1.3.9). I was planning to postpone this to basemap 1.4.0, but this way I can use basemap 1.3.9 as test release to ensure that the conda-forge recipe is working as expected for the next minor release.

A simple example would be as follows:

(base) vic@alpha:~$ conda create -y -c conda-forge -n test39 python=3.9 basemap
Collecting package metadata (current_repodata.json): done
Solving environment: done

==> WARNING: A newer version of conda exists. <==
  current version: 23.3.1
  latest version: 23.11.0

Please update conda by running

    $ conda update -n base -c conda-forge conda

Or to minimize the number of packages updated during conda update use

     conda install conda=23.11.0

## Package Plan ##

  environment location: /opt/homebrew/Caskroom/miniforge/base/envs/test39

  added / updated specs:
    - basemap
    - python=3.9

The following NEW packages will be INSTALLED:

  basemap            conda-forge/osx-arm64::basemap-1.3.9-py39hb118012_0 
  basemap-data       conda-forge/noarch::basemap-data-1.3.2-pyhd8ed1ab_3 
  brotli             conda-forge/osx-arm64::brotli-1.1.0-hb547adb_1 
  brotli-bin         conda-forge/osx-arm64::brotli-bin-1.1.0-hb547adb_1 
  bzip2              conda-forge/osx-arm64::bzip2-1.0.8-h93a5062_5 
  c-ares             conda-forge/osx-arm64::c-ares-1.24.0-h93a5062_0 
  ca-certificates    conda-forge/osx-arm64::ca-certificates-2023.11.17-hf0a4a13_0 
  certifi            conda-forge/noarch::certifi-2023.11.17-pyhd8ed1ab_0 
  contourpy          conda-forge/osx-arm64::contourpy-1.2.0-py39he9de807_0 
  cycler             conda-forge/noarch::cycler-0.12.1-pyhd8ed1ab_0 
  fonttools          conda-forge/osx-arm64::fonttools-4.47.0-py39h17cfd9d_0 
  freetype           conda-forge/osx-arm64::freetype-2.12.1-hadb7bae_2 
  geos               conda-forge/osx-arm64::geos-3.12.1-h965bd2d_0 
  importlib-resourc~ conda-forge/noarch::importlib-resources-6.1.1-pyhd8ed1ab_0 
  importlib_resourc~ conda-forge/noarch::importlib_resources-6.1.1-pyhd8ed1ab_0 
  kiwisolver         conda-forge/osx-arm64::kiwisolver-1.4.5-py39hbd775c9_1 
  krb5               conda-forge/osx-arm64::krb5-1.21.2-h92f50d5_0 
  lcms2              conda-forge/osx-arm64::lcms2-2.16-ha0e7c42_0 
  lerc               conda-forge/osx-arm64::lerc-4.0.0-h9a09cb3_0 
  libblas            conda-forge/osx-arm64::libblas-3.9.0-20_osxarm64_openblas 
  libbrotlicommon    conda-forge/osx-arm64::libbrotlicommon-1.1.0-hb547adb_1 
  libbrotlidec       conda-forge/osx-arm64::libbrotlidec-1.1.0-hb547adb_1 
  libbrotlienc       conda-forge/osx-arm64::libbrotlienc-1.1.0-hb547adb_1 
  libcblas           conda-forge/osx-arm64::libcblas-3.9.0-20_osxarm64_openblas 
  libcurl            conda-forge/osx-arm64::libcurl-8.5.0-h2d989ff_0 
  libcxx             conda-forge/osx-arm64::libcxx-16.0.6-h4653b0c_0 
  libdeflate         conda-forge/osx-arm64::libdeflate-1.19-hb547adb_0 
  libedit            conda-forge/osx-arm64::libedit-3.1.20191231-hc8eb9b7_2 
  libev              conda-forge/osx-arm64::libev-4.33-h93a5062_2 
  libffi             conda-forge/osx-arm64::libffi-3.4.2-h3422bc3_5 
  libgfortran        conda-forge/osx-arm64::libgfortran-5.0.0-13_2_0_hd922786_1 
  libgfortran5       conda-forge/osx-arm64::libgfortran5-13.2.0-hf226fd6_1 
  libjpeg-turbo      conda-forge/osx-arm64::libjpeg-turbo-3.0.0-hb547adb_1 
  liblapack          conda-forge/osx-arm64::liblapack-3.9.0-20_osxarm64_openblas 
  libnghttp2         conda-forge/osx-arm64::libnghttp2-1.58.0-ha4dd798_1 
  libopenblas        conda-forge/osx-arm64::libopenblas-0.3.25-openmp_h6c19121_0 
  libpng             conda-forge/osx-arm64::libpng-1.6.39-h76d750c_0 
  libsqlite          conda-forge/osx-arm64::libsqlite-3.44.2-h091b4b1_0 
  libssh2            conda-forge/osx-arm64::libssh2-1.11.0-h7a5bd25_0 
  libtiff            conda-forge/osx-arm64::libtiff-4.6.0-ha8a6c65_2 
  libwebp-base       conda-forge/osx-arm64::libwebp-base-1.3.2-hb547adb_0 
  libxcb             conda-forge/osx-arm64::libxcb-1.15-hf346824_0 
  libzlib            conda-forge/osx-arm64::libzlib-1.2.13-h53f4e23_5 
  llvm-openmp        conda-forge/osx-arm64::llvm-openmp-17.0.6-hcd81f8e_0 
  matplotlib-base    conda-forge/osx-arm64::matplotlib-base-3.7.3-py39h6685e7c_0 
  munkres            conda-forge/noarch::munkres-1.1.4-pyh9f0ad1d_0 
  ncurses            conda-forge/osx-arm64::ncurses-6.4-h463b476_2 
  numpy              conda-forge/osx-arm64::numpy-1.26.2-py39heee92a0_0 
  openjpeg           conda-forge/osx-arm64::openjpeg-2.5.0-h4c1507b_3 
  openssl            conda-forge/osx-arm64::openssl-3.2.0-h0d3ecfb_1 
  packaging          conda-forge/noarch::packaging-23.2-pyhd8ed1ab_0 
  pillow             conda-forge/osx-arm64::pillow-10.1.0-py39h755f0b7_0 
  pip                conda-forge/noarch::pip-23.3.2-pyhd8ed1ab_0 
  proj               conda-forge/osx-arm64::proj-9.3.1-h93d94ba_0 
  pthread-stubs      conda-forge/osx-arm64::pthread-stubs-0.4-h27ca646_1001 
  pyparsing          conda-forge/noarch::pyparsing-3.1.1-pyhd8ed1ab_0 
  pyproj             conda-forge/osx-arm64::pyproj-3.6.1-py39hd1c2dfb_5 
  pyshp              conda-forge/noarch::pyshp-2.3.1-pyhd8ed1ab_0 
  python             conda-forge/osx-arm64::python-3.9.18-hd7ebdb9_1_cpython 
  python-dateutil    conda-forge/noarch::python-dateutil-2.8.2-pyhd8ed1ab_0 
  python_abi         conda-forge/osx-arm64::python_abi-3.9-4_cp39 
  readline           conda-forge/osx-arm64::readline-8.2-h92ec313_1 
  setuptools         conda-forge/noarch::setuptools-68.2.2-pyhd8ed1ab_0 
  six                conda-forge/noarch::six-1.16.0-pyh6c4a22f_0 
  sqlite             conda-forge/osx-arm64::sqlite-3.44.2-hf2abe2d_0 
  tk                 conda-forge/osx-arm64::tk-8.6.13-h5083fa2_1 
  tzdata             conda-forge/noarch::tzdata-2023d-h0c530f3_0 
  unicodedata2       conda-forge/osx-arm64::unicodedata2-15.1.0-py39h0f82c59_0 
  wheel              conda-forge/noarch::wheel-0.42.0-pyhd8ed1ab_0 
  xorg-libxau        conda-forge/osx-arm64::xorg-libxau-1.0.11-hb547adb_0 
  xorg-libxdmcp      conda-forge/osx-arm64::xorg-libxdmcp-1.1.3-h27ca646_0 
  xz                 conda-forge/osx-arm64::xz-5.2.6-h57fd34a_0 
  zipp               conda-forge/noarch::zipp-3.17.0-pyhd8ed1ab_0 
  zstd               conda-forge/osx-arm64::zstd-1.5.5-h4f39d0f_0 

Downloading and Extracting Packages

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate test39
#
# To deactivate an active environment, use
#
#     $ conda deactivate

And a simple test for this new environment:

(base) vic@alpha:~$ conda activate test39
(test39) vic@alpha:~$ python -c "from mpl_toolkits import basemap; print(basemap.__version__)"
1.3.9

I found out that it is very important for basemap to stay within the conda-forge channel for the dependencies, otherwise there is unexpected behaviour (some numpy dylibs missing).

cc @jucor since you were asking about this very long ago.

jucor commented 11 months ago

Brilliant! Thanks a lot for the update @molinav :)

molinav commented 10 months ago

Hello, I have been trying to run code originally written for Python 2, but I have encountered some issues when attempting to run it

Traceback (most recent call last): File "PyVelest.py", line 27, in from mpl_toolkits.basemap import Basemap File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/mpl_toolkits/basemap/init.py", line 53, in import _geoslib ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_geoslib.so, 0x0002): symbol not found in flat namespace '_GEOSArea'

I have done everything that others have suggested in this issue, but I don't have any idea what's going on. I use a Mac M2, and I don't know if there is any problem with Macs.

@Isaac0880 May I ask you to open a separate issue for the specific problem that you are describing? If you provide a complete traceback and a description of the environment, I can try to see what can be done.