Improve performance through the CyHy environment's SSH bastion

[mcdonnnj]

💡 Summary

We should look at ways to improve performance through the CyHy environment's SSH bastion.

Motivation and context

There are an ever increasing number of users connecting to the CyHy environment's SSH bastion to access the internal database for myriad reasons. It is also the case that service support has difficultly working with entities that have large host counts. I believe performance through the SSH bastion may be the culprit based on information learned when troubleshooting performance in the COOL.

Implementation notes

I believe the most likely culprit is the limited baseline bandwidth available to the CyHy environment's SSH bastion due to its current instance type (t3.small). The networking information for that instance type can be seen here:

$ aws ec2 describe-instance-types \
    --filters "Name=instance-type,Values=t3.small" \
    --query "InstanceTypes[].[ \
        InstanceType, \
        NetworkInfo.NetworkPerformance, \
        NetworkInfo.NetworkCards[0].BaselineBandwidthInGbps]" \
    --output table
------------------------------------------
|          DescribeInstanceTypes         |
+-----------+-------------------+--------+
|  t3.small |  Up to 5 Gigabit  |  0.128 |
+-----------+-------------------+--------+

In addition I checked the metrics for the ENA driver on the currently deployed instance with specific attention to the bw_in_allowance_exceeded and bw_out_allowance_exceeded metrics:

$ sudo ethtool -S ens5 | grep allowance_exceeded
     bw_in_allowance_exceeded: 59247378
     bw_out_allowance_exceeded: 1164
     pps_allowance_exceeded: 0
     conntrack_allowance_exceeded: 0
     linklocal_allowance_exceeded: 0

Acceptance criteria

• [ ] ENA metrics have improved and hopefully as a result so has performance through the CyHy environment's SSH bastion.

[jsf9k]

I think some content is missing after "bw_in_allowance_exceeded and bw_out_allowance_exceeded metrics:".

[mcdonnnj]

I think some content is missing after "bw_in_allowance_exceeded and bw_out_allowance_exceeded metrics:".

Yeah I have comment placeholders. I was typing this on my desktop but I need to pull data from my work laptop to populate two spots.