SSH protocol error. Possible causes include using the wrong key file or username.

[Fuzzy-sh]

• Flintrock version: 11.0
• Python version: 3.5.5 / anaconda
• OS: Amazon Linux 2 AMI (HVM), SSD Volume Type - ami-0b5a47f8865280111
• region of the instance is Singapore

Config.yaml

services:
  spark:
      version: 2.3.3
    # git-commit: latest  # if not 'latest', provide a full commit SHA; e.g. d6dc12ef0146ae409834c78737c116050961f350
    # git-repository:  # optional; defaults to https://github.com/apache/spark
    # optional; defaults to download from from the official Spark S3 bucket
    #   - must contain a {v} template corresponding to the version
    #   - Spark must be pre-built
    #   - must be a tar.gz file
    # download-source: "https://www.example.com/files/spark/{v}/spark-{v}.tar.gz"
      download-source: "https://www.apache.org/dyn/closer.lua?action=download&filename=spark/spark-{v}/spark-{v}-bin-hadoop2.7.tgz"
    # executor-instances: 1
  hdfs:
      version: 2.8.5
    # optional; defaults to download from a dynamically selected Apache mirror
    #   - must contain a {v} template corresponding to the version
    #   - must be a .tar.gz file
    # download-source: "https://www.example.com/files/hadoop/{v}/hadoop-{v}.tar.gz"
    # download-source: "http://www-us.apache.org/dist/hadoop/common/hadoop-{v}/hadoop-{v}.tar.gz"
      download-source: "https://www.apache.org/dyn/closer.lua?action=download&filename=hadoop/common/hadoop-{v}/hadoop-{v}.tar.gz"
provider: ec2

providers:
  ec2:
    key-name: Key-flint
    identity-file: /home/ec2-user/certs/Key-flint.pem
    instance-type: t2.micro
    region: ap-southeast-1
    # availability-zone: <name>
    ami: ami-0b5a47f8865280111  # Amazon Linux 2, us-east-1
    user: ec2-user
    # ami: ami-61bbf104  # CentOS 7, us-east-1
    # user: centos
    # spot-price: <price>
    # vpc-id: <id>
    # subnet-id: <id>
    # placement-group: <name>
    # security-groups:
    #   - group-name1
    #   - group-name2
    # instance-profile-name:
    # tags:
    #   - key1,value1
    #   - key2, value2  # leading/trailing spaces are trimmed
    #   - key3,  # value will be empty
    # min-root-ebs-size-gb: <size-gb>
    tenancy: default  # default | dedicated
    ebs-optimized: no  # yes | no
    instance-initiated-shutdown-behavior: terminate  # terminate | stop
    # user-data: /path/to/userdata/script

launch:
  num-slaves: 1
  # install-hdfs: True
  install-spark: True

debug: false

[Fuzzy-sh]

hello dear @nchammas Thanks for your efforts to solve the issues about Flintrock. I have received this issue, and I appreciate it if you help me to solve it. Many thanks.

[nchammas]

Thanks for sharing your Flintrock config.

1. Can you also share the full launch output of Flintrock itself?
2. Have you ever successfully launched and SSHed into an EC2 instance (separate from Flintrock)? What identity key and username did you use during that successful attempt?

[Fuzzy-sh]

Thanks for answering :) (Can you also share the full launch output of Flintrock itself? )-- > This is the Photo of the output while launching. Even it builds the Instances for me, but after a while, this error appears.

(Have you ever successfully launched and SSHed into an EC2 instance (separate from Flintrock)? What identity key and username did you use during that successful attempt?) -- > Not on the Amazon Linux 2, but the centos yes. The process is quite different, though, too many steps :(. The username was Centos. For a passwordless connection, I create a key in .ssh for both, and it worked.

[Fuzzy-sh]

Is that the same, I should connect to instances build by flintrock and create that ssh connection?

[nchammas]

I recommend that you first confirm you are able to launch and connect to an instance using the same identity key and username, but outside of Flintrock.

So if you are using Key-flint.pem, ami-0b5a47f8865280111, and ec2-user, try first to launch and connect to an instance with these settings using the AWS CLI or the AWS Web Console, without Flintrock involved at all.

Once you have a working combination, update your Flintrock settings accordingly and try using Flintrock again. Let me know if you still have problems at that point.

[Fuzzy-sh]

Thanks a ton, dear @nchammas For sure, I will take your advice. May I ask why you are concerned about launching the instances without Flintrock involved? Did I apply the key mistakenly? Or its a part of the steps should have been done beforehand?

[nchammas]

The SSH protocol error you are seeing is typically caused by some problem with the AMI, identity file, or username you are using. So I want to remove Flintrock from the equation and first confirm that you can use the same combination of AMI, identity file, and username without Flintrock involved, before trying again with Flintrock.

This is just a troubleshooting technique to help us understand what is going wrong. Does that make sense?

[Fuzzy-sh]

Thanks, It does make sense. However, I have seen very few decent tutorials in that regard. I hope I can make it accurate to share a detailed oriented document helpful for novices. :)

[nchammas]

There are a few tutorials for using Flintrock linked to from the README.

If you mean a tutorial for launching a barebones EC2 instance, I'm sure there are several of those readily available online, but I don't have any specific recommendations.

[Fuzzy-sh]

I meant using Flintrock :) Thank you.

[Fuzzy-sh]

Hello dear @nchammas You were right about the key problem. I've applied WinSCP software and transfered key.pem directly to AWS instance ( /home/ec2-user/.ssh/key.pem). Then it worked. :) Thanks a million.

[Fuzzy-sh]

Seems like cannot connect from Southeast However, through ami-0b8d0d6ac70e5750c, flintrock works fine and no such error.

[nchammas]

Are you always connecting to the same VPC and subnet? These issues all seem like connectivity problems that are not specific to Flintrock, but rather are related somehow to your VPC setup.

If you figure out how to launch and connect to an instance outside of Flintrock, then that same configuration should work with Flintrock.

Are you all set here, or would you like additional assistance debugging SSH connectivity issues?

[Fuzzy-sh]

Dear @nchammas First of all, many thanks for your reply.

Now I have two EC2 instances 1- the first one is in N. Virginia with the ami-0b8d0d6ac70e5750c. For this one, No connectivity error. 2- the second one is in Singapore with ami-0b5a47f8865280111. For the second one, I received SSH connectivity while launching flintrock. :( Is that related to ami or as you said it should be VPC?.

[nchammas]

If you are using different AMIs then the required username or SSH key may be different. My recommendation is the same as before: First try to launch an instance with ami-0b5a47f8865280111 in the Singapore region using the Amazon Web Console and connect that way. After you figure out how to get that working, then try again in Flintrock with the same settings.

[Fuzzy-sh]

Hello I have launched an instance with ami-0b5a47f8865280111. in the Singapore region using the Amazon Web Console. It gives me this error

[nchammas]

The screenshot you are showing me is from Flintrock, not from the Web Console. The Web Console looks like this:

The URL is usually something like: https://console.aws.amazon.com/ec2/v2/home

I suggest trying to launch an instance using the web console and connecting to it with plain SSH before involving Flintrock. This will help you understand where the SSH issue is coming from without needing to worry about Flintrock.

[Fuzzy-sh]

Here you are. and sorry for the misunderstanding. Thanks. I will take your advice in that regard.

[Fuzzy-sh]

This is my VPC setup

and subnets

[nchammas]

It seems like your VPC doesn't have an Internet Gateway configured. In other words, it's a private VPC that doesn't have any public IP addresses. (I'm guessing this from the 172. IP addresses shown in your screenshot.)

Can you make sure your VPC has an Internet Gateway attached and that instances launched within it are configured to get public IP addresses?

[nchammas]

Hey @fazish, I'm closing this due to its age. If you're still struggling with your setup and think it's related to Flintrock, feel free to reopen this issue!