Where does setup-network-environment.service come from?

[ghost]

👋

Thank you for this example project. But I wonder, where does setup-network-environment.service come from? Is it https://github.com/kelseyhightower/setup-network-environment or is it some AWS-specific thing?

Is there maybe some other way of getting the IPv4 address for vm.args without the env vars exported by setup-network-environment.service?

[ghost]

Ah, it comes from templates/infra.yml.

As for my second question, I decided to try out the pre_configure_hooks.

[bitwalker]

Yes, if you aren't using setup-network-environment, then pre_configure_hooks and your own shell script which exports the IP address of the interface you want to listen on works. You can also write something like setup-network-environment yourself as a one-shot service.

[ghost]

This ugly little script seems to have worked:

#!/usr/bin/env bash

export HOST_IPV4
export REPLACE_OS_VARS=true

HOST_IPV4=$(
  ifconfig \
    | sed -En 's/127.0.0.1//;s/.*inet (addr:)?(([0-9]*\.){3}[0-9]*).*/\2/p' \
    | grep '^10\.' # only interested in private network
)

[bitwalker]

Yeah how you extract the IP can vary considerably depending on the host OS and environment, number of network interfaces, etc. - which is why I pulled in setup-network-environment for the example, but if you know your platform, then a little bit of shell magic is usually all you need :)

[ghost]

There was actually an issue with setup-network-environment according to https://github.com/kelseyhightower/setup-network-environment/issues/8, although if it's still true, it would've worked exactly as I want it to (get private ip).

But now I'm considering using something like terraform's null_resource to export HOST_IPV4. I'm already using it to write out the ip address of all the nodes in the cluster anyway.

# once the droplets are created, we can collect their ips
# and start the cluster
resource "null_resource" "configure-cluster" {
  count = "${var.nodes_count}"

  connection {
    type = "ssh"
    user = "root"
    private_key = "${file(var.private_key_path)}"
    host = "${element(digitalocean_droplet.app.*.ipv4_address, count.index)}"
  }

  # can actually find out the private ip address this way as well
  provisioner "remote-exec" {
    inline = [
      "echo ${join(",", digitalocean_droplet.app.*.ipv4_address_private)} > node_ips.txt"
    ]
  }

  provisioner "remote-exec" {
    script = "../bin/setup_vm"
  }
}

I wonder if you think distillery's usage with something like terraform is worth a guide?

[bitwalker]

I would prefer to highlight different approaches to things like "how do I set the hostname for the node", rather than build out an entire guide just for Terraform, since the parts that differ from deployment to deployment are almost always going to be things you solve with hooks/custom commands (as far as Distillery is concerned anyway).

I'm also hesitant to add a Terraform guide to the docs, since I would be on the hook to support it, and I personally avoid Terraform when possible (though I do work with it on behalf of clients). The AWS guide is less about AWS or CloudFormation specifically, than it is about illustrating not only how to deploy releases as part of a CI/CD pipeline, but what infrastructure like that looks like and takes to set up to begin with. It's supposed to be a "big picture" guide - and given how much work I do with AWS, it makes it relatively easy for me to maintain. I hope that makes sense - normally I would never turn down offers for more documentation, but in the case of something specific like Terraform, it requires considerable effort to keep in sync with upstream changes.

[ghost]

I hope that makes sense - normally I would never turn down offers for more documentation, but in the case of something specific like Terraform, it requires considerable effort to keep in sync with upstream changes.

Yes, it does make sense. I'll probably write a small gist explaining the steps required to make something like https://github.com/servers-and-stuff/terraform-stuff/tree/master/digitalocean-elixir-simple-cluster

I personally avoid Terraform when possible

I've just started using it and I must say I quite like it. Are there some problems that I'd encounter down the road due to its limitations? Do you prefer using the provider's tools like CloudFormation?

[bitwalker]

I've just started using it and I must say I quite like it. Are there some problems that I'd encounter down the road due to its limitations? Do you prefer using the provider's tools like CloudFormation?

There are certainly some nice things about it, though I find that it is more often than not a leaky abstraction. In particular with AWS it makes little sense to use it over CloudFormation, especially since you can write CF in YAML. CF is easy to work with across a team, just share the YAML, whereas Terraform stores a lot of secret information as well as hardcoded directory names in it's state, which has to be committed to the repository to even share across the team at all, or alternatively, use one of the remote store options. I've wired up the S3 remote store before, and ultimately I regretted it, since it didn't work very well anyway, and requires a bootstrapping process that can be very fragile if you want to automate everything.

The other part that is nice about working directly in CF is that you can do things like I do in this example project - deploy infrastructure as part of the deployment process itself. This means I can tweak the CF, push a new commit with those changes, and the resulting deployment will roll out those infrastructure changes. It just has a very nice integration with other AWS tools/services. While you might be able to achieve something similar with Terraform, it's just way easier using CF. You can also load up the CF in the designer in the CF console, and see all of the resources and how they connect, and make changes or export images, etc.

Outside of AWS, if something like CF is offered (e.g. CDM for GCP), I'll use that - otherwise I'd prefer to use something like Salt personally. That said, I don't think Terraform is terrible or anything, it's just not been worth the extra layer of abstraction in my experience.