search

de-data-lab / census-tract-redistricting

0 stars 0 forks source link

Cannot start the main.py: importing geopandas fails with `ImportError: dlopen(...)` #5

Closed nsunami closed 2 years ago

nsunami commented 2 years ago

I cannot start the main.py after installing dependencies. It seems that importing geopandas is failing. I'm using Mac.

I found a fix discussed on geopanda's repo: https://github.com/conda-forge/geopandas-feedstock/issues/45

Full error log: ``` Traceback (most recent call last): File "/Users/nami/git/census-tract-redistricting/main.py", line 3, in import geopandas File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/geopandas/__init__.py", line 1, in from geopandas._config import options # noqa File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/geopandas/_config.py", line 109, in default_value=_default_use_pygeos(), File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/geopandas/_config.py", line 95, in _default_use_pygeos import geopandas._compat as compat File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/geopandas/_compat.py", line 9, in import pyproj File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pyproj/__init__.py", line 49, in import pyproj.network File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pyproj/network.py", line 10, in from pyproj._network import ( # noqa: F401 pylint: disable=unused-import ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pyproj/_network.cpython-310-darwin.so, 0x0002): tried: '/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pyproj/_network.cpython-310-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/lib/_network.cpython-310-darwin.so' (no such file) ```
mohamadbaksh commented 2 years ago

I just added requirements.txt using anaconda try that out and let me know

nsunami commented 2 years ago

Thanks! But, I cannot use the requirements.txt because it contains paths to environment-specific packages (it seems like).

The paths to the packages seem to be hard-coded like this: file:///home/conda/feedstock_root/build_artifacts/anaconda-client_1619451397123/work

So, running the command pip3 install -r requirements.txt fails:

Processing /home/conda/feedstock_root/build_artifacts/anaconda-client_1619451397123/work
ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/System/Volumes/Data/home/conda/feedstock_root/build_artifacts/anaconda-client_1619451397123/work'
mohamadbaksh commented 2 years ago

I have created a new requirements-linux.txt which worked on my MacOS

nsunami commented 2 years ago

Great! But still not working...

pip in a conda environment

  1. Ran pip3 install -r requiremnts-linux.txt on a conda environment.

  2. Got the following error:

    × Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [1 lines of output]
  proj executable not found. Please set the PROJ_DIR variable. For more information see: https://pyproj4.github.io/pyproj/stable/installation.html
  [end of output]

  3. Tried export PROJ_DIR=/usr/local on terminal

  4. Got a new error:

    × Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [3 lines of output]
  PROJ_DIR is set, using existing PROJ installation..

  ERROR: PROJ_INCDIR dir not found. Please set PROJ_INCDIR.
  [end of output]

Gave up on using pip and moved on to conda install

Creating a virtual environment via virtualenv and try again

I tried creating a virtual environment via virtualenv (not conda) and tried running pip3 install -r requiremnts-linux.txt, but ended up in the same place as above.

Using conda

  1. Ran conda install --file requiremnts-linux.txt

  2. Failed with the following error:

PackagesNotFoundError: The following packages are not available from current channels:

  - shapely==1.8.1.post1
  - click==8.1.0
  - kiwisolver==1.4.1

Current channels:

  - https://conda.anaconda.org/conda-forge/osx-64
  - https://conda.anaconda.org/conda-forge/noarch
  - https://repo.anaconda.com/pkgs/main/osx-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/osx-64
  1. Changed the shapely, click, and kiwisolver to what's available on the conda forge. But a got a lengthy error on dependency (UnsatisfiableError: The following specifications were found to be incompatible with each other: ...)

I'm here now. Maybe a gremlin lives in my environment 😆 . Will continue troubleshooting.

nsunami commented 2 years ago

I solved the issue!

Problem

For some reason, my python alias was leading to /usr/bin/python even within the virtual environment.

Solution

I had to do conda deactivate several times to get out from the base environment. And then conda activate ... to activate the environment.

Checking with which python:

/Users/nami/opt/anaconda3/envs/redistricting/bin/python

Then all is good to go! I still don't know why activating an environment from my base environment was pointing to /usr/bin/python