Installing spatial libraries on AppService (Python with Django)

[oi-eddyoj]

Azure allows the use of Python, Django, Postgres database and even Postgres add-ons such as PostGIS for working with spatial data.

Any webapp should be able to work with spatial data and it's not uncommon. However, libraries typically require the use of external dependencies e.g. GDAL. Please see this page for reference as to what libraries are required:

https://docs.djangoproject.com/en/2.2/ref/contrib/gis/install/geolibs/

It seems like quite an oversite to not think about working with GIS data.

If running a Linux VM, this is trivial as we can run the shell command: sudo apt-get install binutils libproj-dev gdal-bin, as indicatedd in the docs referenced above.

However, when using an Appservice, it is unclear as to how we achieve this.

Any guidance on how we run the shell command mentioned above? Where would the associated packages be installed?

[AdamS-MSFT]

@oi-eddyoj Thank you for the valuable feedback,we are investigating the issue.

[oi-eddyoj]

@Adam-Smith-MSFT I have had to go down the route of creating containers, which has taken me 3 days to learn so far. Docker is a monumental waste of time for small business functions to develop applications with, but it makes sense for much bigger organizations.

The functionality of being able to run custom commands in a linux appservice is so critical in keeping app development cheap. It is very tempting to leave Azure just for this issue.

[BryanTrach-MSFT]

@oi-eddyoj We apologize for the frustration that you have encountered. Our blessed images are locked down a bit more than a IaaS VM image. As a result, there's a bit of a different workaround. Please see the below steps to achieve this.

1. SSH into the container and copy the current startup script by typing "cp /opt/startup/startup.sh /home"
2. Edit the startup.sh under /home/startup.sh and add the following lines to the top of the file after "#!/bin/sh".

    # Installing spatial dependencies

    apt-get update -qq && apt-get install binutils libproj-dev gdal-bin -yqq

3. In the Azure Portal configurations, add "/home/startup.sh" as the Startup Command and restart the site.

We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.

[spechtx]

Hi @BryanTrach-MSFT

Thank you very much for helping with this issue.

I came across the same issue and followed your steps. Unfortunately i get an application error, right after the app starts. When i remove /home/startup.sh" as the Startup Command everything is fine again. Do you have any advice, in how to get the GDAL library working with Django on Azure Web Service?

[askiefer]

^ I think I'm having the same issue as above, commented here: https://github.com/microsoft/Oryx/issues/1049#issuecomment-877492288

@spechtx Were you getting the GDALException error, could not find GDAL library?

[spechtx]

@askiefer yes, this was the error I got:

Could not find the GDAL library (tried "gdal", "GDAL", "gdal3.2.0", "gdal3.1.0", "gdal3.0.0", 
"gdal2.4.0", "gdal2.3.0", "gdal2.2.0", "gdal2.1.0", "gdal2.0.0"). Is GDAL installed? 
If it is, try setting GDAL_LIBRARY_PATH in your settings.

Thank you for commenting on the feature request I posted. Seems like they are working on a solution for the missing libraries.

[askiefer]

Thanks for confirming @spechtx. I am surprised that there is not already support for GDAL. Fingers crossed it is resolved soon!

[jalexmelendez]

We still haven't got support for this, it seems that my build is failing too, this is so frustrating tbh, anyway this is my error message:

raise ImproperlyConfigured(\ndjango.core.exceptions.ImproperlyConfigured: Could not find the GDAL library (tried "gdal", "GDAL", "gdal3.2.0", "gdal3.1.0", "gdal3.0.0", "gdal2.4.0", "gdal2.3.0", "gdal2.2.0", "gdal2.1.0", "gdal2.0.0"). Is GDAL installed? If it is, try setting GDAL_LIBRARY_PATH in your settings. if i can solve this issue i will let you know...

This is a part i modified in my workflow file to install this dependencies, yet i haven't managed to get a successful boot.

`build: runs-on: ubuntu-latest

steps:
  - uses: actions/checkout@v2

  - name: Install gdal-bin
    run: sudo apt-get install binutils libproj-dev gdal-bin

  - name: Set up Python version
    uses: actions/setup-python@v1
    with:
      python-version: '3.9'`

[vanakenm]

Doing the same as @jalexmelendez here, with the same result (the build phase succeed, but the messages about gdal not being found persist).

To make things worse, as this make the container fail at startup there is no way to ssh into it to debug the issue (are the libs there ? are they on a different path than expected, etc)

Any clue? This will prevent a lot of Django app to be deployed on App Service

[jalexmelendez]

@vanakenm If it helps you i managed to deploy my app to Heroku including the PSQL configuration on a migration to add gdal support on it, i can send you the container configuration i used if you want

[vanakenm]

@jalexmelendez this is much appreciated - but my goal is to migrate my app (currently on... heroku :D) to Azure in this case.

[jalexmelendez]

@vanakenm Got it mate! If you manage to deploy it please let me know, you should also try Google Cloud, it is a great service too and i haven't tried to deploy my GeoDjango app there yet...

[vanakenm]

For @jalexmelendez and anyone intersted - I think I managed to make it work.

I used a custom startup command. Trouble was I could not add/update it (as the container failed to start properly, so I end up doing a one liner and put it directly via the Azure configuration screen (Configuration > General Settings > Custom Startup Command)

My command is:

apt-get update && apt-get -y install binutils libproj-dev gdal-bin && gunicorn --bind=0.0.0.0 --timeout 600 app.wsgi

We need the update (else the install fail), the "-y" to avoid the process blocking there and finally we need to add the actual command to run the Django app (make sure to replace app.wsgi with the name of the Django app containing the wsgi.py file).

Hope it can help someone else. @BryanTrach-MSFT maybe useful to put that somewhere in a doc? I may create a quick blog post about it.