Tunnel keeps closing after updating to latest 7777 version

[LasseRafn]

My tunnels closes within 5-15 minutes now, with the following message

|----------------◆-----------|
The tunnel has been closed (the container probably self-terminated after reaching its timeout).
|--------------◆-------------|e container probably self-terminated after reaching its timeout).

I did not have this before updating from 1.1.4 to 1.1.9 (or whichever is the latest version

[mnapoli]

Hey @LasseRafn! Over the last 2 days I did multiple tests (and continued using 7777), left it running for 15 min+, even one hour, but the tunnel was still working.

Could you double check that you are on version 1.1.12? Which OS are you using? Are you getting this every single time?

[buddhaCode]

I have the same problem. But I can not connect to a database at all. My clients get a timeout when I try to connect. I tried Table Plus, Sequel Ace and mysql cli.

Retrieving the availability zone and subnet of your instance.
Checking if 7777 is set up in the AWS account.
7777 needs to update resources in the AWS account, this can take a couple of minutes.
7777 is now set up, let's connect to the database.
Retrieving the container security group.
Retrieving the computer's IP address.
Authorizing x.x.x.x on the security group.
Starting the Fargate container.
The IP address of the container is x.x.x.x.
Starting the SSH tunnel to x.x.x.x:22.

Tunnel created 🎉
Connect to the database on port 7777 on your machine.

Press Ctrl+C to stop and destroy the connection.
SSH tunnel error: The tunnel could not be opened for an unknown reason. Original error message: (SSH) Channel open failure: Operation timed out
The tunnel has been closed (the container probably self-terminated after reaching its timeout).

I am using the lastest version on Mac OS 13.4

7777 --version
7777/1.1.12 darwin-x64 node-v14.20.0

Is there any way to go back to an earlier version? I also tried the --forever switch. Same result.

I also ran 7777 unsinstall and tried it again with the recreated cloud formation stack. Did not work either.

[mnapoli]

@buddhaCode do you see any logs in the Fargate task?

You can download an older version via a URL like this: https://releases.port7777.com/1.1.4/macos/7777 (I'd recommend 1.1.4)

[buddhaCode]

The Fargate Logs look like this:

8.6.2023, 16:57:29 MESZ | connect_to myproject-production.xxxx.eu-central-1.rds.amazonaws.com port 3306: failed. | 7777-bastion
8.6.2023, 16:54:22 MESZ | Accepted publickey for root from x.x.x.x port 51500 ssh2: RSA XXXXX | 7777-bastion
8.6.2023, 16:54:20 MESZ | Server listening on 0.0.0.0 port 22. | 7777-bastion
8.6.2023, 16:54:20 MESZ | Server listening on :: port 22. | 7777-bastion
8.6.2023, 16:54:20 MESZ | Starting the SSH daemon | 7777-bastion
8.6.2023, 16:54:20 MESZ | SSH is starting, the container will terminate in 7200 seconds | 7777-bastion
8.6.2023, 16:54:20 MESZ | Starting | 7777-bastion

I guess there is maybe something wrong with the subnets. But I am not an expert in the network stuff.

Going back to v.1.1.4, the program exits directly after the last step:

Retrieving the availability zone and subnet of your instance.
Checking if 7777 is set up in the AWS account.
7777 is already set up, moving on.
Retrieving the container security group.
Retrieving the computer's IP address.
Authorizing x.x.x.x on the security group.
Starting the Fargate container.
The IP address of the container is x.x.x.x.
Starting the SSH tunnel to x.x.x.x:22.

The Fargate Log with v.1.1.4 looks like this:

8.6.2023, 17:20:49 MESZ | Unable to negotiate with x.x.x.x port 52055: no matching host key type found. Their offer: ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-rsa [preauth] | 7777-bastion
8.6.2023, 17:20:49 MESZ | Timeout: terminating after 7200 seconds | 7777-bastion
8.6.2023, 17:20:48 MESZ | Server listening on 0.0.0.0 port 22. | 7777-bastion
8.6.2023, 17:20:48 MESZ | Server listening on :: port 22. | 7777-bastion
8.6.2023, 17:20:48 MESZ | Starting the SSH daemon | 7777-bastion
8.6.2023, 17:20:48 MESZ | SSH is starting, the container will terminate in 7200 seconds | 7777-bastion
8.6.2023, 17:20:48 MESZ | Starting | 7777-bastion

I am using the serverless vpc plugin to configure the network stuff. Maybe there is an error? I am sure, that it already worked in the past. Here is my vpc plugin config:

        vpcConfig:{
            // Whether plugin is enabled. Can be used to selectively disable plugin
            // on certain stages or configurations. Defaults to true.
            enabled: true,
            cidrBlock: '10.0.0.0/16',
            // if createNatGateway is a boolean "true", a NAT Gateway and EIP will be provisioned in each zone
            // if createNatGateway is a number, that number of NAT Gateways will be provisioned
            createNatGateway: 1,
            // When enabled, the DB subnet will only be accessible from the Application subnet
            // Both the Public and Application subnets will be accessible from 0.0.0.0/0
            createNetworkAcl: true,
            // Whether to create the DB subnet. Default: true
            createDbSubnet: true,
            // Whether to enable VPC flow logging to an S3 bucket
            createFlowLogs: false,
            // Whether to create a bastion host
            createBastionHost: false,
            bastionHostKeyName: 'MyKey', // required if creating a bastion host
            // Whether to create a NAT instance
            createNatInstance: false,
            // Whether to create AWS Systems Manager (SSM) Parameters
            createParameters: false,
            // Optionally specify AZs (defaults to auto-discover all availabile AZs)
            zones: [
                // 'eu-central-1a'
            ],
            // By default, S3 and DynamoDB endpoints will be available within the VPC
            // see https://docs.aws.amazon.com/vpc/latest/userguide/vpc-endpoints.html
            // for a list of available service endpoints to provision within the VPC
            // (varies per region)
            services: [
                // 'kms',
                // 'secretsmanager',
                'rds-data'
            ],
            // Optionally specify subnet groups to create. If not provided, subnet groups
            // for RDS, Redshift, ElasticCache and DAX will be provisioned.
            subnetGroups: ['rds'],
            // Whether to export stack outputs so it may be consumed by other stacks
            exportOutputs: false
        }

My database is created like this:

            AppSecurityGroupEgressRDS: {
                Type: 'AWS::EC2::SecurityGroupEgress',
                Properties: {
                    Description: 'permit MySQL (3306) to DBSecurityGroup',
                    DestinationSecurityGroupId: {'Ref': 'DBSecurityGroup'},
                    FromPort: 3306,
                    ToPort: 3306,
                    GroupId: { 'Fn::GetAtt': ['AppSecurityGroup', 'GroupId'] },
                    IpProtocol: 'tcp'
                }
            },
            DBSecurityGroup: {
                Type: 'AWS::EC2::SecurityGroup',
                Properties: {
                    GroupDescription: 'RDS Security group for ${self:service}-${sls:stage}',
                    SecurityGroupEgress: [{
                        Description: 'deny all outbound',
                        IpProtocol: '-1',
                        CidrIp: '127.0.0.1/32'
                    }],
                    SecurityGroupIngress: [{
                        Description: 'permit MYSQL(3306) from AppSecurityGroup',
                        IpProtocol: 'tcp',
                        FromPort: 3306,
                        ToPort: 3306,
                        SourceSecurityGroupId: { 'Fn::GetAtt': ['AppSecurityGroup', 'GroupId'] },
                    }],
                    Tags: [
                        {
                            Key: 'Name',
                            Value: { 'Fn::Join': ['-', [{'Ref': 'AWS::StackName'}, 'rds']] }
                        }
                    ],
                    VpcId: {'Ref': 'VPC'}
                }
            },
            DBInstance: {
                Type: 'AWS::RDS::DBInstance',
                DeletionPolicy : 'Retain',
                Properties: {
                    AllocatedStorage: '45', //25 /45
                    AllowMajorVersionUpgrade : false,
                    AutoMinorVersionUpgrade : true,
                    AvailabilityZone : 'eu-central-1a',
                    BackupRetentionPeriod : 7,
                    CACertificateIdentifier : 'rds-ca-2019',
                    CopyTagsToSnapshot : true,
                    DBInstanceClass : 'db.t3.small',
                    DBInstanceIdentifier : '${self:service}-${sls:stage}',
                    DBName : '${self:service}',
                    DBSnapshotIdentifier : '',
                    DBSubnetGroupName : {'Ref': 'RDSSubnetGroup'},
                    DeletionProtection : true,
                    Engine : 'mysql',
                    EngineVersion : '8.0.32',
                    LicenseModel : 'general-public-license',
                    MasterUsername : '${ssm:/${self:service}/${sls:stage}/db/user}',
                    MasterUserPassword : '${ssm:/${self:service}/${sls:stage}/db/password}',
                    MultiAZ : false,
                    NetworkType : 'IPV4',
                    Port : '3306',
                    PreferredBackupWindow : '20:06-20:36',
                    PreferredMaintenanceWindow : 'sun:18:03-sun:18:33',
                    PubliclyAccessible : false,
                    StorageEncrypted : true,
                    StorageType : 'gp2',
                    VPCSecurityGroups : [
                        { 'Fn::GetAtt': ['DBSecurityGroup', 'GroupId'] },
                    ]
                }
            },

[deleugpn]

It seems there is a connectivity issue between your Fargate Container and your RDS as pointed out by this log entry:

8.6.2023, 16:57:29 MESZ | connect_to myproject-production.xxxx.eu-central-1.rds.amazonaws.com port 3306: failed. | 7777-bastion

It could either be your subnet, your security group or your Network ACL. 7777 Attempts to choose the first subnet that has internet access, provisions a Security Group for the Container and attach it to the RDS to allow connectivity between the Container and the RDS, however if there are explicit instructions on your VPC to deny such access, then the tunnel cannot be established.

[mnapoli]

After discussing this more, we highly suspect this config line:

            // When enabled, the DB subnet will only be accessible from the Application subnet
            // Both the Public and Application subnets will be accessible from 0.0.0.0/0
            createNetworkAcl: true,

I use the VPC plugin a lot, but never create ACLs. These are advanced features that may be useful for specific scenarios (larger teams, etc.), but here are redundant with the security provided by security groups. In this case, the ACL might be blocking the 7777 connection.

If you have the opportunity, I would recommend disabling the line and redeploying (to remove the ACL), and then testing 7777 again?

[buddhaCode]

That's the catch. When I set the ACLs to false it works like charm. Thanks a lot @deleugpn and @mnapoli for the quick support.