bitwalker
commented
6 years ago @jfrolich Thanks for the feedback!
It supports Distributed Erlang, as the hosts for the application run in the same subnet, so traffic between those hosts is unrestricted. That said, the example application is not built as a clustered application, so we don't really exercise that here, and we're only spinning up one host out of the box, rather than two or more.
While it is nice to have a boilerplate for deployment in AWS, and automating stuff as much as possible, this is really a complex example, and I would say inaccessible to 99% people.
The infrastructure itself is not very complex, it is really standard stuff, however, there is complexity in the automation provided by CloudFormation, and the use of Lambda/Custom Resources to handle bootstrapping the infrastructure with secrets. There are good reasons for these choices though:
- Security is critical, and this example spins up infra that is correctly isolated (ALBs, hosts, and database all get their own subnets, with all traffic disallowed except what is expressly permitted). The way secrets are handled are an important part of this - we avoid storing any secrets locally, in AWS, or in transit, in unencrypted form - the only place they get decrypted is in the application itself.
- Any real world application in AWS is going to need a load balancer, this sets that up, using ALBs, which support protocols that the classic ELBs do not, i.e. future-proof
- Beyond the infrastructure itself, solving the question of "how do I actually deploy to this infra?" is a big part of this guide - we set up the pieces which make that as simple as it would be to deploy to Heroku, i.e.
git push. Doing these things by hand is failure prone to say the least, and needs to be synchronized with database migrations and infrastructure changes. This sets up all of that. - Automated infrastructure changes - by using CloudFormation, when such changes are performed, they are done in the right order, and made part of the deployment process itself. By placing the infrastructure definition alongside the code itself, changes to the infra get code reviewed just like any other change.
- Since the infrastructure is defined in code, it makes it possible for me to write this guide, and for you or anybody else to clone the repo, change a predefined set of values, and spin up the exact same infrastructure.
While most people new to AWS, or just unexposed to all of the components used in this guide, will not understand it at first contact, I don't believe that is important, nor the point. The first purpose of this guide is to follow along and see what is possible, given that 99% of the guide is automated, the actual time required is very short to go from zero to fully-working example.
The second is to see what a real-world, modern architecture in AWS looks like - for many people, they don't even know where to begin trying to set up something which matches their experience with providers like Heroku. This demonstrates what that looks like, as well as how easy it is to replicate once you've done the hard work of designing the architecture and translating that to CloudFormation.
Lastly, it is expected that readers who want to pursue using this architecture in production, will work backwards to understand each of the components (e.g. CodePipeline, or CloudFormation) at a high level, in order to better understand how things are set up, and how they can go about modifying things to their own tastes.
It's my opinion that too many guides try to "keep it simple", and ultimately fail to show you anything of interest - they leave too much out of the picture. The architecture in this guide contains a lot of the hard stuff, but automates all of it, so you can choose to delve further as you want to tweak some part of the system (say by introducing a manual approval step into the build/deploy pipeline, or by spinning up a clustered application rather than a simple Phoenix app). It's not a "one size fits all" architecture of course, but there is no such thing - the point is that this example demonstrates enough of the hard parts that you could build your own thing from scratch and still have a useful reference for some of the parts in common.
Perhaps it would be better for the community to write a guide of the bare bones fundamental stuff that need to be set up in order to deploy to AWS
These guides do exist already, they just don't have an "official" place in the guides in Distillery's docs. I'm not fundamentally opposed to adding more AWS guides, particularly if they cover a different approach - but I do wonder if the promise of not being complicated really holds true in practice. Even a simple setup with a load balancer, an EC2 host, and a database involves spinning up a VPC, an internet gateway, subnets for those three pieces, plus the network rules. You get to omit the deployment automation part by just having readers scp the release to the EC2 host, but you have to cover Docker builds as well, since that is no longer implicitly handled by the build pipeline. There is also the issue of secrets, even if it is a manual step, you still have to demonstrate it in the guide. By the end, the guide is quite long, and the more steps there are, the more likely the reader is to make a mistake by missing a step, or entering the wrong value somewhere.
... and the advanced secret management that this thing implements.
Should I describe this in more detail in the guide? Or have a section at the end which introduces each of the components used in more detail?
Just that this didn't really help me and I already deployed Elixir in AWS before. Actually I am considering building this with a guide + docker image.
I'm sorry about that :(. Can you tell me what would've made this guide more useful to you, despite the complexity? Perhaps more description of what things are used and why, etc.? I definitely want to make it accessible and useful, while maintaining the architecture as designed, at least for this guide; but it is a first draft (i.e. I wrote the project, then wrote the guide), so I'm sure there is a lot of room for improvement.
jfrolich
rlefevre
philihp
Nice work! I looked at all the scripts and configuration, but as far as I could see this doesn't support distributed erlang, or am I wrong?
Some other feedback: While it is nice to have a boilerplate for deployment in AWS, and automating stuff as much as possible, this is really a complex example, and I would say inaccessible to 99% people.
Perhaps it would be better for the community to write a guide of the bare bones fundamental stuff that need to be set up in order to deploy to AWS (it's not very complicated). And then have a special section that goes into advanced stuff, like automating the setup of infra and code-deploys, and the advanced secret management that this thing implements.
Not saying that this should be it, or putting out a request. Just that this didn't really help me and I already deployed Elixir in AWS before. Actually I am considering building this with a guide + docker image.