Support VPC networking mode

[k2nr]

Barcelona is currently using bridge mode for container networking where container instances have only one ENI that is associated with its host network. Right now the bridge networking mode has many limitations.

• Tasks can't have dedicated security groups
• VPC flow logs only shows host network
• ECS Service Discovery can't be used
  • bridge network is supported but ECS Service Discovery uses SRV DNS record instead of A records which is not quite useful.
• with bridge mode the same task definition can't be used for Fargate because Fargate doesn't support bridge network
• NLB can't be used
• lots more

bridge network mode has been working well but it is also blocking us from supporting many new features so we should migrate to VPC mode

[k2nr]

I researched this a little bit. There is a limitation on ENI count per EC2 instance:

Each task that uses the awsvpc network mode receives its own elastic network interface, which is attached to the container instance that hosts it. EC2 instances have a limit to the number of elastic network interfaces that can be attached to them, and the primary network interface counts as one. For example, a c4.large instance may have up to three elastic network interfaces attached to it. The primary network adapter for the instance counts as one, so you can attach two more elastic network interfaces to the instance. Because each awsvpc task requires an elastic network interface, you can only run two such tasks on this instance type. For more information about how many elastic network interfaces are supported per instance type, see IP Addresses Per Network Interface Per Instance Type in the Amazon EC2 User Guide for Linux Instances.

In our normal use case we use t2.medium which can have 3 ENIs which means each instance can have only 2 containers that use vpc networking mode.

I don't think this is our option until this problem is solved somehow.

UPDATE ENI trunking is released. the above problem is solved if we use an instance type that supports ENI trunking