AWS service discovery integration

[morgo]

Please answer these questions before submitting your issue. Thanks!

1. What did you do?

AWS is the largest public cloud, but the Kubernetes support is still immature (and costs $$$ for a small install).

It would be nice to have pd/tidb/tikv integrate with Route53's service discovery out of the box. This can be done with the AWS GO SDK.

2. What did you expect to see?

It would be nice to be able to do something like this (for first run):

./bin/pd-server \
 --name=pd1 \
 --data-dir=pd \
 --aws-bootstrap
Created new Route53 Service "srv-iy3d7hhlf5cjciph".

And then for subsequent:

./bin/pd-server \
 --name=pd1 \
 --data-dir=pd \
 --aws-service-id=srv-iy3d7hhlf5cjciph

Then with TiDB and TiKV:

./bin/tidb-server \
--store=tikv \
--aws-service-id=srv-iy3d7hhlf5cjciph

./bin/tikv-server \
--aws-service-id=srv-iy3d7hhlf5cjciph \
 --data-dir=tikv

3. What did you see instead?

Instead it requires complex shell scripting because of the multiple commands:

aws servicediscovery create-private-dns-namespace --vpc vpc-b9ed28c2 --name tidb.local
(capture the output of the operation id)
(check to get the namespace id from an api command using the operation-id, waiting in a loop)
aws servicediscovery create-service --name pd --namespace-id <value-here>

INTERNALIP=`curl http://169.254.169.254/latest/meta-data/local-ipv4`
INSTANCEID=`curl http://169.254.169.254/latest/meta-data/instance-id`

screen -dm ./pd-master-linux-amd64/bin/pd-server \
 --name=pd1 \
 --data-dir=pd \
 --client-urls="http://$INTERNALIP:2379" \
 --peer-urls="http://$INTERNALIP:2380" \
 --initial-cluster="pd1=http://$INTERNALIP:2380" \
 --log-file=pd.log

aws servicediscovery register-instance --service-id srv-iy3d7hhlf5cjciph --instance-id $INSTANCEID --attributes=AWS_INSTANCE_IPV4=$INTERNALIP

4. What version of PD are you using (pd-server -V)?

[gregwebs]

I think PD should definitely facilitate bootstrap more, but not necessarily in a cloud specific way. Its nice to try to keep pd dependencies light and not ship a dependency on every cloud vendor. Maybe part of this could be a cloud provider neutral enhancement and part shell script? I think the shell script is doable, I am used to writing those for AWS and could help with that.

I already have a suggestion for getting rid of the TIKV advertise-addr argument: https://github.com/tikv/tikv/issues/4447

[morgo]

I think the shell script is doable, I am used to writing those for AWS and could help with that.

I would prefer either built-in, or another go binary. Shell scripts are hard to make portable since only the language (bash) is constant. All the programs can vary. Some time back MySQL switched from using shell and perl scripts to only C++ binaries, and it's made installation smoother.

[gregwebs]

Its true that the aws cli for example does make breaking changes. I agree it is nice to have shell script at the most just download the actual programs that have well-specified dependencies. But they are good for prototyping solutions.