Heroku deploy crashes, due to PostGIS

[beechnut]

Upon deploying, the application crashes (see logs below), seemingly because of a NewRelic issue.

2014-12-18T16:51:31.131198+00:00 heroku[api]: Release v6 created by cloydster@gmail.com
2014-12-18T16:51:31.550951+00:00 heroku[web.1]: State changed from crashed to starting
2014-12-18T16:51:36.715894+00:00 heroku[web.1]: Starting process with command `newrelic-admin run-program gunicorn --chdir src planbox.heroku_wsgi:application --bind 0.0.0.0:37976 --worker-class gevent --workers 4 --config gunicorn.conf.py`
2014-12-18T16:51:37.593322+00:00 app[web.1]: bash: newrelic-admin: command not found
2014-12-18T16:51:38.322538+00:00 heroku[web.1]: State changed from starting to crashed
2014-12-18T16:51:38.315116+00:00 heroku[web.1]: Process exited with status 127

The new-relic command should be available because it's in requirements.txt, though Heroku started out detecting this as a Node app, not Python.

remote: Compressing source files... done.
remote: Building source:
remote:
remote: -----> Node.js app detected
remote: -----> Resetting git environment
remote: -----> Requested node range:  0.10.x
remote: -----> Resolved node version: 0.10.33

Looking through the logs, I don't think it installs requirements.txt.

[beechnut]

Following the instructions in this StackOverflow article I set it explicitly as a Python app by running

heroku config:set BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-python

I pushed the branch again to make it build as a Python app.

It installed dependencies, but crashed with this stack trace.

heroku[web.1]: Starting process with command `newrelic-admin run-program gunicorn --chdir src planbox.heroku_wsgi:application --bind 0.0.0.0:59355 --worker-class gevent --workers 4 --config gunicorn.conf.py`
heroku[web.1]: State changed from starting to up
app[web.1]: Traceback (most recent call last):
app[web.1]:   File "/app/.heroku/python/bin/gunicorn", line 9, in <module>
app[web.1]:     load_entry_point('gunicorn==19.0.0', 'console_scripts', 'gunicorn')()
app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 74, in run
app[web.1]:     WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/app/base.py", line 166, in run
app[web.1]:     super(Application, self).run()
app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/app/base.py", line 71, in run
app[web.1]:     Arbiter(self).run()
app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/arbiter.py", line 196, in run
app[web.1]:     self.halt(reason=inst.reason, exit_status=inst.exit_status)
app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/arbiter.py", line 292, in halt
app[web.1]:     self.stop()
app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/arbiter.py", line 343, in stop
app[web.1]:     time.sleep(0.1)
app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/arbiter.py", line 209, in handle_chld
app[web.1]:     self.reap_workers()
app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/gunicorn/arbiter.py", line 459, in reap_workers
app[web.1]:     raise HaltServer(reason, self.WORKER_BOOT_ERROR)
app[web.1]: gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
heroku[web.1]: State changed from up to crashed
heroku[web.1]: Process exited with status 1

[beechnut]

I lowered the number of workers in the Procfile to 1, because we're trying to fly under the paid threshold for the moment.

I re-pushed the code, and was still getting errors. Then I ran heroku run python src/manage.py shell and received an error about the geometry type, which reminded me I hadn't set up PostGIS (though thought the heroku pg:transfer heroku-pg-transfer would have taken care of that.

It turns out PostGIS, which planbox depends on, is not available on development-tier databases.

[beechnut]

PostGIS was only part of the problem. When deploying to Heroku, set BUILDPACK_URL to https://github.com/ddollar/heroku-buildpack-multi.git. This enables the use of the .buildpacks file in the repository to get all of the frameworks necessary to run PostGIS and GeoDjango.

[beechnut]

After re-deploying, it still crashes and when I run the shell it errors out thusly:

$ heroku run python src/manage.py shell

Running `python src/manage.py shell` attached to terminal... up, run.8586
Traceback (most recent call last):
  File "src/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
    utility.execute()
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute
    django.setup()
  File "/app/.heroku/python/lib/python2.7/site-packages/django/__init__.py", line 21, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
    app_config.import_models(all_models)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/apps/config.py", line 197, in import_models
    self.models_module = import_module(models_module_name)
  File "/app/.heroku/python/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/auth/models.py", line 40, in <module>
    class Permission(models.Model):
  File "/app/.heroku/python/lib/python2.7/site-packages/django/db/models/base.py", line 125, in __new__
    new_class.add_to_class('_meta', Options(meta, **kwargs))
  File "/app/.heroku/python/lib/python2.7/site-packages/django/db/models/base.py", line 300, in add_to_class
    value.contribute_to_class(cls, name)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/db/models/options.py", line 166, in contribute_to_class
    self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
  File "/app/.heroku/python/lib/python2.7/site-packages/django/db/__init__.py", line 40, in __getattr__
    return getattr(connections[DEFAULT_DB_ALIAS], item)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/db/utils.py", line 242, in __getitem__
    backend = load_backend(db['ENGINE'])
  File "/app/.heroku/python/lib/python2.7/site-packages/django/db/utils.py", line 108, in load_backend
    return import_module('%s.base' % backend_name)
  File "/app/.heroku/python/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/gis/db/backends/spatialite/base.py", line 10, in <module>
    from django.contrib.gis.db.backends.spatialite.introspection import SpatiaLiteIntrospection
  File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/gis/db/backends/spatialite/introspection.py", line 1, in <module>
    from django.contrib.gis.gdal import OGRGeomType
  File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/gis/gdal/__init__.py", line 46, in <module>
    from django.contrib.gis.gdal.driver import Driver  # NOQA
  File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/gis/gdal/driver.py", line 5, in <module>
    from django.contrib.gis.gdal.prototypes import ds as capi
  File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/gis/gdal/prototypes/ds.py", line 8, in <module>
    from django.contrib.gis.gdal.libgdal import lgdal
  File "/app/.heroku/python/lib/python2.7/site-packages/django/contrib/gis/gdal/libgdal.py", line 48, in <module>
    lgdal = CDLL(lib_path)
  File "/app/.heroku/python/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libjpeg.so.62: cannot open shared object file: No such file or directory

I added PIL to try to fix the libjpeg error with no luck, following the instructions here but using the git-based PIL repo.

[mjumbewu]

Regarding the original problem, I'm not sure what was happening there. It looks like newrelic is listed in requirements.txt, so it should have been installed.

Regarding the libjpeg.so stuff, yeah, this is a nasty issue and I'm not exactly sure why it happens yet. It appears to be some sort of caching issue with how Heroku handles buildpacks. I ran into this with the Shareabouts API and started a convo about it on the heroku-geo-buildpack repo: https://github.com/cyberdelia/heroku-geo-buildpack/issues/22#issuecomment-66656881. My "solution" there was to change the geo-buildpack reference in .buildpacks to https://github.com/cyberdelia/heroku-geo-buildpack.git#e1b845b. Hopefully a more reliable solution will be found soon.