Enable BBR TCP Congestion Control Algorithm out of the box

[sidewinder12s]

Is there a reason this is not enabled out of the box?

I saw another issue that appeared to indicate it's enabled but its still not on the latest AMI.

v20241011:

$ sysctl net.ipv4.tcp_available_congestion_control
net.ipv4.tcp_available_congestion_control = reno cubic

[ndbaker1]

were you referring to https://github.com/awslabs/amazon-eks-ami/issues/432? In it, the author already mentions how they enabled BBR, but i think its not going to show up until the kmod is explicitly loaded, like -

[root@ip-192-168-143-4 bin]# sysctl net.ipv4.tcp_available_congestion_control
net.ipv4.tcp_available_congestion_control = reno cubic
[root@ip-192-168-143-4 bin]# modinfo tcp_bbr
filename:       /lib/modules/6.1.102-108.177.amzn2023.x86_64/kernel/net/ipv4/tcp_bbr.ko
description:    TCP BBR (Bottleneck Bandwidth and RTT)
license:        Dual BSD/GPL
author:         Soheil Hassas Yeganeh <soheil@google.com>
author:         Yuchung Cheng <ycheng@google.com>
author:         Neal Cardwell <ncardwell@google.com>
author:         Van Jacobson <vanj@google.com>
srcversion:     52FE9DC9CC96C6188A84576
depends:
retpoline:      Y
intree:         Y
name:           tcp_bbr
vermagic:       6.1.102-108.177.amzn2023.x86_64 SMP preempt mod_unload modversions
sig_id:         PKCS#7
signer:         Amazon Linux Kernel Signing Key
sig_key:        68:CE:0B:E9:C8:08:61:0D:28:E6:EE:2D:B9:7C:51:F7:0B:2F:41:5D
sig_hashalgo:   sha256
signature:      22:AF:C1:0F:FF:A5:C8:68:AE:A0:4B:14:E9:F8:10:BF:25:CE:C4:4A:
                4A:0C:EC:BE:90:5F:29:CB:AF:F9:68:FC:4C:C0:53:24:CB:28:A9:52:
                6D:11:FD:19:40:DC:73:D9:8C:DA:FA:D4:0F:48:30:4A:38:B7:6E:41:
                00:90:57:92:52:28:03:1E:31:27:6A:9A:94:21:AD:E5:03:52:41:19:
                F3:B5:81:DA:B6:FE:E6:02:B3:4E:FA:3C:D3:51:56:B5:4A:1A:6A:74:
                F2:EB:68:2C:9F:5B:95:28:A2:7C:B5:B1:1A:64:88:A6:7C:C5:3A:52:
                6C:80:5A:32:32:65:77:CA:DD:CA:B8:87:50:5F:D6:AA:EB:84:A7:1B:
                9C:35:6C:B1:88:70:3F:AC:D8:0D:E1:A5:09:41:39:BE:D5:F9:50:8E:
                A6:93:F3:F3:A5:41:C1:4F:F7:04:95:52:C5:2F:E9:53:85:D4:BE:4D:
                59:6E:3B:D5:1F:E2:C8:35:40:EF:16:97:F0:56:47:DD:44:3D:97:80:
                AF:CF:0B:7C:0E:EA:6A:D1:DB:15:B6:7B:15:0E:87:C9:24:8C:F9:AB:
                EF:49:CA:B9:EE:33:83:8F:0B:71:86:55:2B:B1:B3:28:25:CC:9E:4D:
                64:0C:6F:A2:EE:F3:FC:28:BC:A7:1A:71:5F:21:BF:8D:20:19:69:11:
                6B:7F:74:64:D9:89:0F:CA:C1:21:0A:F0:A9:9B:FF:D6:78:5A:E2:17:
                80:08:64:2A:23:FE:07:10:B2:33:31:E1:D7:DE:37:86:2D:5B:A0:A7:
                65:53:EE:32:E6:CC:22:0B:62:93:76:8C:63:23:81:C1:96:0F:5C:64:
                53:06:7A:D6:D9:DD:51:6F:94:0D:17:04:3F:9F:2D:C9:80:69:7C:1E:
                5F:10:9B:1D:E4:F5:BC:66:59:EA:AE:0B:AB:D8:4E:B5:D5:C2:F5:26:
                FC:87:EB:3B:91:17:9F:D6:C0:B3:C9:57:31:8A:38:35:C6:AC:AF:94:
                AC:97:68:D5:7F:01:83:10:28:F6:4F:DF:8B:FF:60:89:78:EC:FC:BB:
                9A:BD:43:FB:1D:8B:7A:AE:FE:D2:8A:EF:28:E8:34:D0:09:D3:05:A2:
                BF:3B:D6:4D:05:B3:00:BF:A7:3C:71:05:60:11:CE:38:E2:CF:90:E9:
                02:D1:44:15:E2:A1:F7:B0:B5:00:53:2A:11:70:CC:DB:D9:02:EB:86:
                DB:9B:EA:70:1B:5E:D1:81:1E:C2:A1:17:4E:A7:51:6E:FA:B3:79:A1:
                3C:76:7D:68:16:C4:C5:02:7E:CD:AE:69:BB:7F:C8:17:22:8A:09:86:
                0F:EA:49:0E:F4:81:6E:58:18:DE:60:CE
[root@ip-192-168-143-4 bin]# modprobe tcp_bbr
[root@ip-192-168-143-4 bin]# sysctl net.ipv4.tcp_available_congestion_control
net.ipv4.tcp_available_congestion_control = reno cubic bbr

[sidewinder12s]

I mean, I had read that ticket like BBR was already enabled out of the box for the AMI. It is not from what I was checking. So this ticket is linked/asking can we just make it the default.

[ndbaker1]

I think that's a conscious decision the end-user should make based on observed network metrics. BBR might conflict with fairness guarantees from the other congestion control algorithms, so deviating from the kernel's default seems likely to confuse IMO

[cartermckinnon]

Changing the default on AL2 at this point is not something we're likely to do, but I'm not opposed to doing this on AL2023.

[sidewinder12s]

I think that's a conscious decision the end-user should make based on observed network metrics. BBR might conflict with fairness guarantees from the other congestion control algorithms, so deviating from the kernel's default seems likely to confuse IMO

I see (and looking up some benchmarking of BBR)

[sidewinder12s]

So what prompted this ask was that we were seeing microbursting on the ENA was leading to dropped traffic so we were looking for potential mitigations in the EKS AMI, VPC CNI or elsewhere that might reduce the impact of these microbursts.