search

hackoregon / devops-17

deployment tools for Hack Oregon projects
4 stars 3 forks source link

Enable an optimized docker image that is acceptable for projects requiring GeoJSON/GIS Data #23

Closed BrianHGrant closed 7 years ago

BrianHGrant commented 7 years ago

Emergency Response and possibly other teams will need to serve GeoJSON data from a PostGIS database. To do so project will require the use of GeoDjango and django-rest-framework-gis.

As per GeoDjango documentation, the GEOS and GDAL Geospatial libraries are required to handle and convert PostGIS into GeoJSON. My current implementation of this is working, the docker image takes about 20 minutes for build time as it is compiling the libraries from source. Wondering if there is a better optimization.

My current dev repo is located here . Slack me for the db info @brianhgrant

sanjuroj commented 7 years ago

Thanks for posting this, the transportation team has the exact same need.

znmeb commented 7 years ago

I've got a PostGIS build on top of the Docker Hub official PostgreSQL image at https://github.com/hackoregon/postgis-geocoder-test/tree/master/postgis. It's working but I haven't figured out how to get it to persist data - if you shut it down it re-initializes on startup. Maybe we can merge the two images or I can rebuild on yours.

The geocoder requires PostgreSQL 9.6 and PostGIS 2.3; earlier versions don't use the latest TIGER/Line shapefiles. I know there's a way to persist using data volumes or host partitions mounted on container volumes but I don't want to build something that's not deployable to AWS.

BrianHGrant commented 7 years ago

Shapefiles are something I am still struggling with. I am not sure how to handle/serve or if the frontend will need them vs. the info being served through geometry fields.

For the data persistence, Have you looked at docker volumes?

Docker Volumes

Handled in the docker-compose with:

volumes:
  - .:/code
znmeb commented 7 years ago

Yeah, that was my next step but I want to sync up with DevOps and the other projects. How are you getting PostgreSQL?

As far as I know the front ends are all expecting APIs delivering GeoJSON, not raw shapefiles. So we want the GIS info in the database(s) in a format the API needs. If you've got shapefiles you can get them into PostGIS two ways:

1. shp2pgsql - ships with PostGIS
2. ogr2ogr - ships with GDAL
BrianHGrant commented 7 years ago

The db is live on AWS. In a earlier iteration I had been working with this docker file originally, building of the official postgresql docker image to create a postgis sql image and templating db with proper extensions.

znmeb commented 7 years ago

Ah ... IIRC that's PostgreSQL 9.5. PostGIS 2.3 will probably run on it but it's not a recommended configuration. Does your app need geocoding at run time, for data entry or neither?

BrianHGrant commented 7 years ago

Using this image: https://hub.docker.com/r/brianhgrant/hacko-geodjango/ with github: https://github.com/BrianHGrant/hacko-geodjango-docker

Project build time decrease to 2 minutes and change. Prebuilding the container is way to go. Closing.

MikeTheCanuck commented 7 years ago

That is FANTASTIC news. It's frustrating enough waiting a few minutes for a Travis build, let alone the interminable 20 you were enduring. Nice work!