mojodna / marblecutter-virtual

Virtual catalogs for marblecutter
Other
48 stars 30 forks source link

Use AWS SAM for deployment #6

Closed mojodna closed 5 years ago

mojodna commented 5 years ago

The Serverless Application Model makes deployment a bit less magical, as IAM policies and other resources are explicitly defined in template.yaml. It also removes a layer of indirection when deployed using Up (the golang proxy server) while also facilitating improved timeout handling. serverless-wsgi (from the Serverless project) is used as a replacement shim to facilitate implementation as a Flask app.

The custom GDAL + rasterio build has been removed in this PR in favor of using wheels; it was the most brittle part of the whole process, forcing specific versions of Python libraries to be used. As there were some performance improvements available from it (HTTP/2 support, libjpeg-turbo decompression improvements), I hope to resurrect it soon as a Lambda Layer (esp. once the SAM CLI supports building apps against layers, rather than just simulating requests).

This also changes the Lambda Python runtime to 3.6 (from 2.7) while keeping the Docker Compose configuration against 2.7 to facilitate version compatibility checks. In future, we probably want to update Docker Compose to use 3.6 (or 3.7).

mojodna commented 5 years ago

I've put together a draft layer (based on deps/ here), but haven't tried using it yet: https://github.com/mojodna/lambda-layer-rasterio