etcd-wrapper: Not a good option for gateway use

[maikzumstrull]

Feature Request

Environment

GCE and bare metal

Desired Feature

It seems to me the desired architecture for a larger Container Linux cluster is to run etcd-member.service on some arbitrarily chosen nodes, and have etcd gateway running everywhere else - so that applications can transparently reach etcd on localhost:2379 on any node.

etcd-member uses /usr/lib/coreos/etcd-wrapper to launch etcd in a rkt container.

For the gateway use case, this seems suboptimal to me for a few reasons:

• The script currently makes ETCD_DATA_DIR mandatory. A gateway doesn't need a data dir.
• This pulls in a lot of etcd code a gateway doesn't need (Raft, gRPC, etc)
• It means an extra image download on every machine, for base OS features (locksmithd) to work

Overall, I think:

• etcd gateway should be a separate, much smaller binary, not an etcd subcommand
• This binary should ship in the base image

[glevand]

This seems reasonable to me. Maybe have a condition in etcd-wrapper to run etcd gateway, then the user would create a systemd plug-in triggering the condition on each system that is to use etcd gateway.

[bloo]

I ran into this today as well. I'd also like to be able to configure additional nodes that will run the gateway by explicitly sending them the initial cluster list - I can only seem to get it working by sending the discovery URL (which isn't ideal if you're to add nodes later). It was easier to set proxy: on using etcd v2. @maikzumstrull any workarounds? cc @glevand

[dghubble]

@bloo there is a etcd-gateway example among the matchbox reference clusters. https://github.com/coreos/matchbox/blob/master/examples/ignition/etcd3-gateway.yaml

- name: etcd-member.service
  enable: true
  dropins:
    - name: 40-etcd-cluster.conf
      contents: |
        [Service]
        Environment="ETCD_IMAGE_TAG=v3.2.0"
        ExecStart=
        ExecStart=/usr/lib/coreos/etcd-wrapper gateway start \
          --listen-addr=127.0.0.1:2379 \
          --endpoints=node1.example.com:2379,node2.example.com:2379,node3.example.com:2379

[bloo]

@dghubble missed that - thanks!

[dghubble]

WRT the original request:

Even if etcd decided to split etcd-gateway into a separate binary / image (they're not right now), an aim has been to reduce the binaries shipped in the base image. They make the base image larger. The container images (and associated wrapper scripts) allow pulling images that are needed and additionally allow the etcd version to be bumped independent of the OS.

A hypothetical etcd-gateway binary might shave a bit off the ~35MB etcd image you pull in your situation, but would increase the overall image size ~15MB for everyone.