GDAL for Cedar-14 stack should be built against libjpeg8, not libjpeg62

[mjumbewu]

Thanks for maintaining this buildpack! I have gotten a lot of use out of it over the last year or so.

The new cedar-14 stack ships with libjpeg8, not libjpeg62 (see https://devcenter.heroku.com/articles/cedar-ubuntu-packages). But the GDAL downloaded by default for heroku-geo-buildpack appears to be compiled against libjpeg62. For a quick confirmation of this, you can run:

git clone https://github.com/mjumbewu/super-simple-django-starter.git
cd super-simple-django-starter
git checkout geo
heroku create
heroku config:add BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-multi.git
git push heroku geo:master
heroku run python app.py shell

Once you're in the shell, run:

>>> from django.contrib.gis import gdal

You'll get a traceback to the tune of:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/gis/gdal/__init__.py", line 46, in <module>
    from django.contrib.gis.gdal.driver import Driver  # NOQA
  File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/gis/gdal/driver.py", line 5, in <module>
    from django.contrib.gis.gdal.prototypes import ds as capi
  File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/gis/gdal/prototypes/ds.py", line 8, in <module>
    from django.contrib.gis.gdal.libgdal import lgdal
  File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/gis/gdal/libgdal.py", line 48, in <module>
    lgdal = CDLL(lib_path)
  File "/app/.heroku/python/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libjpeg.so.62: cannot open shared object file: No such file or directory

[cyberdelia]

Did you setup the buildpack?

heroku config:add BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-multi.git

[cyberdelia]

Oh, I can reproduce, it seems that there is a problem with the STACK value.

[cyberdelia]

Looks like this goes away for some reason if you do another release ಠ̯ಠ

[mjumbewu]

So did you get it to work? I'm still running into the same results with a clean application.

[mjumbewu]

I see what you're saying about the STACK value. I forked the buildpack to turn on debug and I got some weird behavior. Here are the first few lines of my deploy when debug is off:

-----> Fetching custom git buildpack... done
-----> Multipack app detected
=====> Downloading Buildpack: https://github.com/mjumbewu/heroku-dummy-buildpack.git
=====> Detected Framework: geos/gdal/proj
       Using geos version: 3.4.1
       Using gdal version: 1.11.0
       Using proj version: 4.8.0
-----> Fetching and vendoring geos
...

Notice the GDAL version is 1.11.0. The cedar-14 GDAL that @cyberdelia has on S3 is 1.11.1, so this is pulling (I assume) from the cedar folder.

Then I turned on debug (uncommented set -x in the compile script). Here are the corresponding first few lines of the subsequent deployment:

-----> Fetching custom git buildpack... done
-----> Multipack app detected
=====> Downloading Buildpack: https://github.com/mjumbewu/heroku-dummy-buildpack.git
=====> Detected Framework: geos/gdal/proj
+ unset GIT_DIR
+ S3_BUCKET=cyberdelia-geo-buildpack
+ BUILD_DIR=/tmp/build_b964495b33b6ddf4d860d32a821748f8
+ CACHE_DIR=/app/tmp/cache/cedar-14
+ PROFILE_PATH=/tmp/build_b964495b33b6ddf4d860d32a821748f8/.profile.d/geo.sh
++ dirname /tmp/build_b964495b33b6ddf4d860d32a821748f8/.profile.d/geo.sh
+ mkdir -p /tmp/build_b964495b33b6ddf4d860d32a821748f8/.profile.d
++ manifest_version geos
++ curl http://cyberdelia-geo-buildpack.s3.amazonaws.com/cedar-14/manifest.geos -s -o -
++ head -n 1
+ GEOS_VERSION=3.4.2
++ manifest_version gdal
++ curl http://cyberdelia-geo-buildpack.s3.amazonaws.com/cedar-14/manifest.gdal -s -o -
++ head -n 1
+ GDAL_VERSION=1.11.1
++ manifest_version proj
++ curl http://cyberdelia-geo-buildpack.s3.amazonaws.com/cedar-14/manifest.proj -s -o -
++ head -n 1
+ PROJ_VERSION=4.8.0_1
+ echo 'Using geos version: 3.4.2'
+ indent
+ c='s/^/       /'
+ case $(uname) in
++ uname
+ sed -u 's/^/       /'
       Using geos version: 3.4.2
+ indent
+ c='s/^/       /'
+ case $(uname) in
+ echo 'Using gdal version: 1.11.1'
++ uname
+ sed -u 's/^/       /'
       Using gdal version: 1.11.1
+ echo 'Using proj version: 4.8.0_1'
+ indent
+ c='s/^/       /'
+ case $(uname) in
++ uname
+ sed -u 's/^/       /'
       Using proj version: 4.8.0_1
+ VENDORED_GEOS=vendor/geos/3.4.2
+ VENDORED_GDAL=vendor/gdal/1.11.1
+ VENDORED_PROJ=vendor/proj/4.8.0_1
+ mkdir -p /app/tmp/cache/cedar-14
+ '[' '!' -d /app/tmp/cache/cedar-14/vendor/geos/3.4.2 ']'
+ echo '-----> Fetching and vendoring geos'
-----> Fetching and vendoring gdal
...

The interesting thing to note is that the script is now finding GDAL version 1.11.1 (the GEOS and Proj4j versions are different too). If I now switch back to the release with debug off, the behavior is consistent as before.

I'm now not sure if this is a geo-buildpack bug or a Heroku platform bug.

[mjumbewu]

I got this to work by using the commit hash to refer to the version instead of the tag (i.e., https://github.com/cyberdelia/heroku-geo-buildpack.git#e1b845b instead of https://github.com/cyberdelia/heroku-geo-buildpack.git#1.2), even though they're the same thing. Makes me think it's some caching issue. Maybe this is what you meant about making a new release ಠ̯ಠ.

[cyberdelia]

@mjumbewu I also suspect a weird caching issue.

[jamescw]

Im hitting this on dokku, tried the tag trick but same result

[t16n]

I belive this is the same issue...

In my .buildpack:

https://github.com/cyberdelia/heroku-geo-buildpack.git#1.2
https://github.com/heroku/heroku-buildpack-python.git#v54

And yes I tried the e1b845b trick :-1:

Stacktrace:

 File "/app/.heroku/python/lib/python3.4/site-packages/django/contrib/gis/gdal/__init__.py", line 46, in <module>
    from django.contrib.gis.gdal.driver import Driver  # NOQA
  File "/app/.heroku/python/lib/python3.4/site-packages/django/contrib/gis/gdal/driver.py", line 5, in <module>
    from django.contrib.gis.gdal.prototypes import ds as capi
  File "/app/.heroku/python/lib/python3.4/site-packages/django/contrib/gis/gdal/prototypes/ds.py", line 8, in <module>
    from django.contrib.gis.gdal.libgdal import lgdal
  File "/app/.heroku/python/lib/python3.4/site-packages/django/contrib/gis/gdal/libgdal.py", line 48, in <module>
    lgdal = CDLL(lib_path)
  File "/app/.heroku/python/lib/python3.4/ctypes/__init__.py", line 351, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libjpeg.so.62: cannot open shared object file: No such file or directory

[francisvierboom]

Just a quick note for anyone getting this error but for libjpeg.so.8 - I had a working app but strangely started getting this similar error just the other day. Fixed by upgrading to cedar-14.

heroku run "python manage.py check" 
...
  File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/gis/gdal/prototypes/ds.py", line 8, in <module>
    from django.contrib.gis.gdal.libgdal import lgdal
  File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/gis/gdal/libgdal.py", line 48, in <module>
    lgdal = CDLL(lib_path)
  File "/app/.heroku/python/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libjpeg.so.8: cannot open shared object file: No such file or directory

[cyberdelia]

Pin to the new version 1.3, this add support for stack, I've just forgot to tag it. This should fix it for everybody.

[t16n]

Hi,

I am sorry but the Tag still didn't help with the issue:

    from django.contrib.gis.gdal.libgdal import lgdal
  File "/app/.heroku/python/lib/python3.4/site-packages/django/contrib/gis/gdal/libgdal.py", line 48, in <module>
    lgdal = CDLL(lib_path)
  File "/app/.heroku/python/lib/python3.4/ctypes/__init__.py", line 351, in __init__
    self._handle = _dlopen(self._name, mode)