Fixes to enable Flintrock to work when an EC2 instance is used as a h…

nchammas / flintrock

A command-line tool for launching Apache Spark clusters.

Apache License 2.0

636 stars 116 forks source link

Fixes to enable Flintrock to work when an EC2 instance is used as a h… #326

Closed rlaabs closed 3 years ago

rlaabs commented 3 years ago

…ost.

This PR makes the following changes:

in core.provision_cluster I changed the get_ssh_client call host to cluster.master_ip
changed the health check to use the IP instead of the host

I tested this PR by launching a cluster with Spark 2.2.0, HDFS 2.7.0

Fixes #317.

nchammas commented 3 years ago

Ran some acceptance tests against this branch successfully. Merging this in.

I think in a future PR it would be worth going through how we configure or call services (hostname vs. IP; private vs. public address) and make sure we have a consistent and coherent approach throughout.