Open argutierrez00 opened 3 years ago
Hi! Thanks for the detailed report, just a question to be sure:
Starting the SSH tunnel to <container_ip>.
The connection to the SSH tunnel timed out. Please reinstall 7777 or check your Security Group.
Is that instantaneous? Or is the tunnel working for a few minutes before the error happens?
Questions on top of my head: are you running with any kind of VPN or proxy? (I'm wondering about whether the authorized IP address is correct) But then what is surprising is that the tunnel seems to happen, before it fails with a "timeout".
Could you also confirm the version: 7777 --version
Is that instantaneous? Or is the tunnel working for a few minutes before the error happens?
Starting the SSH tunnel to <container_ip>.
), it takes about 15~25 seconds then the last line on the previous log appears for the timeout. I also tried to do 7777 uninstall
, it usually fails and I'll just manually cleanup some remaining ecs tasks that are running, then manually delete the cloudformation stack.➜ ~ 7777 --version 7777/1.0.7 linux-x64 node-v14.4.0
Could you check that your IP address was successfully added to the Security Group of the Bastion? By going to: CloudFormation > port7777 -> Resources -> ContainerSecurityGroupFor{VpcId}, you'll be able to click on a link that takes you to the EC2 Security Group details. Can you confirm that your IP Address is present?
Have you checked that your RDS is receiving a new Security Group attachment to allow the container to connect to it?
Also, could you confirm that you don't have any NACL at the VPC level (route tables) REJECTING port 22 connections?
When you say uninstalling 7777 is failing, can you provide the reason why it fails (CloudFormation stack delete error?) Perhaps understanding why it's not being able to tear down could indicate something that went wrong on the setup.
Could you check that your IP address was successfully added to the Security Group of the Bastion? By going to: CloudFormation > port7777 -> Resources -> ContainerSecurityGroupFor{VpcId}, you'll be able to click on a link that takes you to the EC2 Security Group details. Can you confirm that your IP Address is present?
Have you checked that your RDS is receiving a new Security Group attachment to allow the container to connect to it?
All traffic | All | All | sg- |
---|---|---|---|
MYSQL/Aurora | TCP | 3306 | sg- |
Also, could you confirm that you don't have any NACL at the VPC level (route tables) REJECTING port 22 connections?
When you say uninstalling 7777 is failing, can you provide the reason why it fails (CloudFormation stack delete error?) Perhaps understanding why it's not being able to tear down could indicate something that went wrong on the setup.
2021-04-16 20:07:31 UTC+0800 | 7777Cluster | DELETE_FAILED | Resource handler returned message: "Error occurred during operation 'DeleteClusters SDK Error: The Cluster cannot be deleted while Tasks are active. (Service: AmazonECS; Status Code: 400; Error Code: ClusterContainsTasksException; Request ID: |
---|
On the ECS side, the 7777ClusterUpdate
cluster is still running with 4 tasks:
Task Definition: [INACTIVE] 7777-bastion:3 Last Status: Running
I usually stop those running tasks, the delete the cluster, then delete the CF stack again for the cleanup.
The failure to uninstall can be avoided by issuing 7777 stop
(https://github.com/whilenull/7777-support/blob/main/commands.md#7777-stop) before running 7777 uninstall
. Unfortunately it doesn't seem related to the connectivity issue.
You mentioned your RDS is running on a private subnet. Does your Route Table have a route for target local
associated with all your private subnets?
Does your RDS security group have any explicit rule denying access?
Does your public subnets also have a Route Table with a route targeting local
?
You mentioned your RDS is running on a private subnet. Does your Route Table have a route for target local associated with all your private subnets?
local
in them.Does your RDS security group have any explicit rule denying access?
Inbound: MYSQL/Aurora | TCP | 3306 | sg- |
– |
---|
Outbound:
All traffic | All | All | 0.0.0.0/0 | – |
---|
Does your public subnets also have a Route Table with a route targeting local?
local
as well. Though the RDS is not associated to the public subnets, and currently, the public subnet has no nat gateway/nat instance enabled at the moment. Though I don't think this would affect it as well as I tried it with an enabled nat instance earlier with no luck.Yeah, I also thought about NAT, but it indeed doesn't seem relevant at this point. Your computer (A) is suppose to connect to the bastion (B) and the bastion connects to the RDS (C). B is in a public subnet with an Internet Gateway and C is in a private subnet, but is capable of communicating with B through the local
route.
Everything seems to be correctly set from the networking side. Short from asking for VPC Flow Logs, the only other thing I can think of is checking the Task CloudWatch Logs to see if the sshd
service is failing somehow.
Do you have a firewall on your local network that could be blocking the connection from being established?
Nope, no firewall on my current network.
Also, re: the sshd
logs on the task, the log seems empty too. I checked on the Task > Logs tab, and then the cloudwatch log group port7777-7777LogGroup-
Also, this AWS account was created via AWS Organizations. Might not matter at all, just a hail mary of some sort. :D
I'm getting the same error but only on github actions, not in my computer
Run 7777 --region=us-east-1 --database staging-database-agosto & The connection to the SSH tunnel timed out. Please reinstall 7777 or check your Security Group.
I'm another unfortunate victim to the above. I've checked everything as per the recommendations above, even the cloudwatch logs. On face value it appears that everything is configure and starting as it should. Eeek.
I've found the problem usually stems from networking issues. I experienced this too, but I realised my issue was that I was behind a private VPN which was obscuring the correct source IP.
Hi guys,
I keep encountering the ff: error:
The connection to the SSH tunnel timed out. Please reinstall 7777 or check your Security Group.
Any additional help is aprpeciated.