search

natcap / invest

InVEST®: models that map and value the goods and services from nature that sustain and fulfill human life.
Apache License 2.0
166 stars 68 forks source link

Homebrew-installed libraries interfere with InVEST mac binaries #10

Closed phargogh closed 3 years ago

phargogh commented 4 years ago

When testing out the InVEST 3.8.0 release binaries, I could launch and run the models by reaching into the application and running the compiled invest binary. But then if I tried running a model that loaded geometries from a vector, the model would segfault. The segfault exception when n_workers=-1 looked like this:

Current thread 0x0000700010e0e000 (most recent call first):
  File "env/lib/python3.7/site-packages/shapely/geometry/polygon.py", line 494 in geos_polygon_from_py
  File "env/lib/python3.7/site-packages/shapely/geometry/polygon.py", line 240 in __init__
  File "env/lib/python3.7/site-packages/shapely/geometry/geo.py", line 19 in box
  File "env/lib/python3.7/site-packages/taskgraph/Task.py", line 1071 in _call
  File "env/lib/python3.7/site-packages/taskgraph/Task.py", line 389 in _task_executor
  File "env/lib/python3.7/threading.py", line 870 in run
  File "env/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "env/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x0000700010408000 (most recent call first):
  File "env/lib/python3.7/threading.py", line 296 in wait
  File "env/lib/python3.7/threading.py", line 552 in wait
  File "env/lib/python3.7/site-packages/taskgraph/Task.py", line 1231 in join
  File "env/lib/python3.7/site-packages/taskgraph/Task.py", line 713 in join
  File "env/lib/python3.7/site-packages/natcap/invest/scenic_quality/scenic_quality.py", line 458 in execute
  File "env/lib/python3.7/site-packages/natcap/invest/ui/model.py", line 1645 in _logged_target
  File "env/lib/python3.7/site-packages/natcap/invest/ui/execution.py", line 68 in run
  File "env/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "env/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x000000010b383dc0 (most recent call first):
  File "src/natcap/invest/cli.py", line 561 in main
  File "src/natcap/invest/cli.py", line 575 in <module>
Segmentation fault: 11

After digging into this some more, it turned out that my homebrew install had GEOS 3.7.3 and GDAL 3.0.1 installed due to a homebrew-managed QGIS install, and that the InVEST binaries were clearly loading the GEOS binaries from the homebrew Cellar (verified by sampling the files that the invest process had open, visible in Activity Monitor). See the attached sample log for more information.

My questions around this are:

  1. why was GEOS being loaded from the Cellar and not from the bundled binaries?
  2. how can we prevent this from happening in the future?

It would appear that pyinstaller modifies LD_LIBRARY_PATH ... maybe that has something to do with this?

phargogh commented 4 years ago

Homebrew-installed proj (osgeo/osgeo4mac/osgeo-proj) also interferes with the binaries we ship, at least on my mac. See #108

phargogh commented 4 years ago

Homebrew-installed geos also seems to be an issue.

phargogh commented 4 years ago

Note to self: working on this over on https://github.com/phargogh/invest/tree/bugfix/10-homebrew-interferes-with-pyinstaller

phargogh commented 3 years ago

Closing after #358