Deployment - moving to Docker?

[IsmailM]

Further to a few questions about deployment:

Current Setup

We do not use Docker at all for production.

At the moment, we are using Elastic Beanstalk for deploying. In the future, we may want to move to docker - i.e. with AWS ECS or the docker option with Elastic beanstalk.

We are currently using the default python-3 image generated by AWS EB. This environment is set up with gunicorn as the default WSGI server. See more here: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create-deploy-python-container.html.

With Elastic Beanstalk - we only provide this repo. And it is able to use the configuration files (inside .platform and .elasticbeanstalk) to install any dependencies - and it then launches gunicorn using the procfile in the repo.

Moving to Docker

It should be relatively easy to move to docker with Elastic beanstalk. The benefit of this would be that we have exactly the same environment for local dev and production

I still think it is a good idea to use AWS RDS for the database, as such, we would only use the app Dockerfile and not the docker-compose.

Current Infrastructure setup

On a side note, here is a description of our infrastructure setup (see image below)

We have a created a custom VPC setup which is used by both the dev and production servers.

The VPC is setup with three private and three public subnets (one subnet per availability zone).

• Essentially the webservers + database are inside the private subnets.
• A load balancer + a bastion server are placed in the public subnet
• These webservers are only accessible from the load balancer. All HTTP/HTTPS traffic is routed to the load balancer (which is in the public subnet), and from there, to a web server.
• The database is only accessible by the web servers (i.e. only within the private subnet).
• The bastion server (in the public subnet) is able to access all servers/database in the private subnets. The bastion server has port 22 open to Neuromancer.

The webservers have autoscaling setup - 1 x t3.small as a base (this is a reserved instance that is paid upfront for 3 years). and as required additional spot instance are added.

The database is managed by AWS RDS (which includes daily backups etc.)

[alanwilter]

@IsmailM The AWS link for EB you posted mentioned python 3.4 but I'm quite sure we are running python 3.6 or higher otherwise it would not be working. Could you please Ismail confirm which version of Python and Gunicorn is running under AWS EB?

[IsmailM]

Well, it seems like the docs are outdated.

We are using python 3.7.8 + gunicorn (version 20.0.4) 😄

For future reference, on the app web servers:

• the virtual env: /var/app/venv/staging-LQM1lest/bin
• source code: /var/app/current

Annoyingly, the IP address of the web-servers always change - due to the auto-scaling adding/removing web-servers.

[pontikos]

@IsmailM what is the latest with this? We haven't moved to docker yet right for deployment as far as I know.

[IsmailM]

We are currently using Elastic Beanstalk.

There are two steps to this:

1. Use Docker within Elastic Beanstalk.
2. Move entirely to Terraform for a cloud-agnostic based deployment stratergy