Closed codingjoe closed 6 years ago
@codingjoe did setting GOES_LIBRARY_PATH
fix the issue for you?
@kennethreitz nope, when connecting to a run dyno yes. But post_compile
is than only available during build, not during runtime. I'm investigating...
Scratch that, it did not work. Hm... strange....
Reading the Django source, I believe GOES_LIBRARY_PATH
needs to be set in Django settings rather than directly in the environment (though likely worth setting that via os.environ
/ the django-environ package):
https://github.com/django/django/blob/1.11.2/django/contrib/gis/geos/libgeos.py
Strange that it's still not working out of the box however.
Fwiw I tried comparing the directory structure of the cedar-14 and heroku-16 geos archives (eg https://lang-python.s3.amazonaws.com/heroku-16/libraries/vendor/geos.tar.gz), and they are identical - so the difference isn't say an inadvertent change in location.
I believe the GEOS libraries aren't functional on heroku-16
, even though they're identical copies to the working ones on cedar-14
. The binary needs to be rebuilt for heroku-16, but i'm having trouble doing so. Might take another approach, possibly (adding it to the build stack image).
@kennethreitz yes, that would be cool. I'd love to switch to heroku-16
I think I figured out a way to run the build. Executing now. Will post an update if it succeeds.
Hi @kennethreitz, did you have a chance to investigate this further? If I'm not mistaken, this is an issue for all GeoDjango installations on Heroku. I'm happy to have a look - could you share your progress so far?
This should be fixed shortly, as soon as the build is complete!
Build complete!
✨🍰✨
Good stuff, thanks Kenneth!
To get GeoDjango working I also had to set the following Config Variables:
GDAL_LIBRARY_PATH=/app/.heroku/vendor/lib/libgdal.so
GEOS_LIBRARY_PATH=/app/.heroku/vendor/lib/libgeos_c.so
Shouldn't the buildpack take care of this when BUILD_WITH_GEO_LIBRARIES=1
is set?
@harelw on which stack?
heroku-16
I'll have to experiment with that myself. If true, the buildpack can definitely do that for you.
@kennethreitz I just checked, it seems the binaries on your bucked have not be updated. This is odd, since I updated all packages in my last 3 pull-requests.
root@8bc20bf16f82:/app# ls -lah /app/.heroku/vendor/lib/
total 85M
drwx------ 4 44182 44182 4.0K Sep 30 14:37 .
drwxr-xr-x 6 root root 4.0K Sep 30 14:37 ..
drwxr-xr-x 2 7527 7527 4.0K Sep 8 19:26 gdalplugins
-rwxr-xr-x 1 7527 7527 1.1K Sep 8 19:26 libgdal.la
lrwxrwxrwx 1 7527 7527 17 Sep 8 19:26 libgdal.so -> libgdal.so.1.18.5
lrwxrwxrwx 1 7527 7527 17 Sep 8 19:26 libgdal.so.1 -> libgdal.so.1.18.5
-rwxr-xr-x 1 7527 7527 62M Sep 8 19:26 libgdal.so.1.18.5
-rwxr-xr-x 1 28988 28988 20M Sep 8 19:17 libgeos-3.4.3.so
-rwxr-xr-x 1 28988 28988 941 Sep 8 19:17 libgeos.la
lrwxrwxrwx 1 28988 28988 16 Sep 8 19:17 libgeos.so -> libgeos-3.4.3.so
-rwxr-xr-x 1 28988 28988 980 Sep 8 19:17 libgeos_c.la
lrwxrwxrwx 1 28988 28988 18 Sep 8 19:17 libgeos_c.so -> libgeos_c.so.1.8.3
lrwxrwxrwx 1 28988 28988 18 Sep 8 19:17 libgeos_c.so.1 -> libgeos_c.so.1.8.3
-rwxr-xr-x 1 28988 28988 1.1M Sep 8 19:17 libgeos_c.so.1.8.3
-rw-r--r-- 1 root root 311K Sep 30 14:37 libjasper.so
-rw-r--r-- 1 root root 311K Sep 30 14:37 libjasper.so.1
-rw-r--r-- 1 root root 311K Sep 30 14:37 libjasper.so.1.0.0
-rwxr-xr-x 1 44182 44182 919 Sep 8 19:15 libproj.la
lrwxrwxrwx 1 44182 44182 16 Sep 8 19:15 libproj.so -> libproj.so.0.7.0
lrwxrwxrwx 1 44182 44182 16 Sep 8 19:15 libproj.so.0 -> libproj.so.0.7.0
-rwxr-xr-x 1 44182 44182 1.3M Sep 8 19:15 libproj.so.0.7.0
drwx------ 2 44182 44182 4.0K Sep 8 19:15 pkgconfig
I ran the build pack in a docker container to debug it, but you can also just download the binaries from your bucket:
curl -O https://lang-python.s3.amazonaws.com/heroku-16/libraries/vendor/gdal.tar.gz
Ok, when I set the pathes manually in my Django application it works:
GDAL_LIBRARY_PATH = "/app/.heroku/vendor/lib/libgdal.so"
GEOS_LIBRARY_PATH = "/app/.heroku/vendor/lib/libgeos_c.so"
I will try what happens if I set the LD_LIBRARY_PATH
to /app/.heroku/vendor/lib
Ok, I opened a pull-request. It should fix the issue for the latest Django releases. The problem with the wrong binaries still remains. I guess you guys should double check your deployment pipeline for those binaries.
Still seems to fail if I remove GDAL_LIBRARY_PATH=/app/.heroku/vendor/lib/libgdal.so
from config vars:
Traceback (most recent call last): File "manage.py", line 23, in <module> execute_from_command_line(sys.argv) File "/app/.heroku/python/lib/python3.6/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line utility.execute() File "/app/.heroku/python/lib/python3.6/site-packages/django/core/management/__init__.py", line 338, in execute django.setup() File "/app/.heroku/python/lib/python3.6/site-packages/django/__init__.py", line 27, in setup apps.populate(settings.INSTALLED_APPS) File "/app/.heroku/python/lib/python3.6/site-packages/django/apps/registry.py", line 85, in populate app_config = AppConfig.create(entry) File "/app/.heroku/python/lib/python3.6/site-packages/django/apps/config.py", line 94, in create module = import_module(entry) File "/app/.heroku/python/lib/python3.6/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 978, in _gcd_import File "<frozen importlib._bootstrap>", line 961, in _find_and_load File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 655, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 678, in exec_module File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed File "/app/.heroku/python/lib/python3.6/site-packages/mapwidgets/__init__.py", line 4, in <module> from .widgets import GooglePointFieldWidget, GooglePointFieldInlineWidget, \ File "/app/.heroku/python/lib/python3.6/site-packages/mapwidgets/widgets.py", line 4, in <module> from django.contrib.gis.forms import BaseGeometryWidget File "/app/.heroku/python/lib/python3.6/site-packages/django/contrib/gis/forms/__init__.py", line 3, in <module> from .fields import ( # NOQA File "/app/.heroku/python/lib/python3.6/site-packages/django/contrib/gis/forms/fields.py", line 4, in <module> from django.contrib.gis.geos import GEOSException, GEOSGeometry File "/app/.heroku/python/lib/python3.6/site-packages/django/contrib/gis/geos/__init__.py", line 5, in <module> from .collections import ( # NOQA File "/app/.heroku/python/lib/python3.6/site-packages/django/contrib/gis/geos/collections.py", line 11, in <module> from django.contrib.gis.geos.geometry import GEOSGeometry, LinearGeometryMixin File "/app/.heroku/python/lib/python3.6/site-packages/django/contrib/gis/geos/geometry.py", line 11, in <module> from django.contrib.gis import gdal File "/app/.heroku/python/lib/python3.6/site-packages/django/contrib/gis/gdal/__init__.py", line 28, in <module> from django.contrib.gis.gdal.datasource import DataSource File "/app/.heroku/python/lib/python3.6/site-packages/django/contrib/gis/gdal/datasource.py", line 39, in <module> from django.contrib.gis.gdal.driver import Driver File "/app/.heroku/python/lib/python3.6/site-packages/django/contrib/gis/gdal/driver.py", line 5, in <module> from django.contrib.gis.gdal.prototypes import ds as vcapi, raster as rcapi File "/app/.heroku/python/lib/python3.6/site-packages/django/contrib/gis/gdal/prototypes/ds.py", line 9, in <module> from django.contrib.gis.gdal.libgdal import GDAL_VERSION, lgdal File "/app/.heroku/python/lib/python3.6/site-packages/django/contrib/gis/gdal/libgdal.py", line 45, in <module> % '", "'.join(lib_names) django.core.exceptions.ImproperlyConfigured: Could not find the GDAL library (tried "gdal", "GDAL", "gdal2.1.0", "gdal2.0.0", "gdal1.11.0", "gdal1.10.0", "gdal1.9.0"). Is GDAL installed? If it is, try setting GDAL_LIBRARY_PATH in your settings.
Yes, same here, I set it manually too.
Wait @harelw actually this should have fixed it: #478
Haven't tried it yet, since it was just merged ;)
Yea I mentioned it because I am pretty sure I tried it after the merge.
I fixed it today with this changes:
So I further drilled own into the issue and found that the Django problem is here: https://github.com/heroku/stack-images/issues/118
I'm getting an
ImportError
from Django when switching to the heroku-16 stack.After some digging, I don't see the difference. The libraries are installed and the
LD_LIBRARY_PATH
is still the same as it used to be on cedar-14.I could fix the issue for me by setting
GEOS_LIBRARY_PATH
explicitly in thepost_compile
hook.bin/post_compile