adobe / aquarium-fish

Your best secure distributed heterogeneous dynamic compute resource manager for CI
Other
7 stars 3 forks source link

AWS: Scrubbing process needs default VPC #71

Open sparshev opened 3 months ago

sparshev commented 3 months ago

If your AWS project doesn't have the default VPC you will see in the logs, the error on triggering scrubbing process, something like that:

Jun 19 01:46:21 <HOST> aquarium-fish[4052331]: 2024/06/19 01:46:21.182268 dedicated_pool.go:303: DEBUG:        AWS: dedicated "test_pool": Dealing with hosts to release: [h-0e69a590017f06c2b]
Jun 19 01:46:21 <HOST> aquarium-fish[4052331]: 2024/06/19 01:46:21.182376 util.go:273: DEBUG:        AWS: Looking an image for type: mac2.metal
Jun 19 01:46:21 <HOST> aquarium-fish[4052331]: 2024/06/19 01:46:21.330845 util.go:285: DEBUG:        AWS: Looking an image for type: found arch: arm64_mac
Jun 19 01:46:21 <HOST> aquarium-fish[4052331]: 2024/06/19 01:46:21.330881 util.go:291: DEBUG:        AWS: Looking an image: Checking past year from 2024
Jun 19 01:46:21 <HOST> aquarium-fish[4052331]: 2024/06/19 01:46:21.830828 util.go:331: DEBUG:        AWS: Found image for specified type "mac2.metal" (arch arm64_mac): ami-002f736bd1ecf4ed5
Jun 19 01:46:21 <HOST> aquarium-fish[4052331]: 2024/06/19 01:46:21.830865 util.go:582: INFO:        AWS: scrubbing h-0e69a590017f06c2b: Selected image: "ami-002f736bd1ecf4ed5"
Jun 19 01:46:22 <HOST> aquarium-fish[4052331]: 2024/06/19 01:46:22.015786 util.go:602: ERROR:        AWS: scrubbing h-0e69a590017f06c2b: Unable to run instance: operation error EC2: RunInstances, https response error StatusCode: 400, RequestID: <RID>, api error VPCIdNotSpecified: No default VPC for this user. GroupName is only supported for EC2-Classic and default VPC.
Jun 19 01:46:22 <HOST> aquarium-fish[4052331]: 2024/06/19 01:46:22.015860 dedicated_pool.go:362: ERROR:        AWS: dedicated "test_pool": Unable to run scrubbing for host "h-0e69a590017f06c2b": AWS: scrubbing h-0e69a590017f06c2b: Unable to run instance: operation error EC2: RunInstances, https response error StatusCode: 400, RequestID: <RID>, api error VPCIdNotSpecified: No default VPC for this user. GroupName is only supported for EC2-Classic and default VPC.

Workaround

You need to create the default VPC in your AWS project. Go to AWS Console --> VPC --> Your VPCs --> Click Actions menu --> Select "Create Default VPC".

Explanation

In order to trigger the scrubbing process Aquarium Fish Allocating the new instance on the dedicated host and then Terminate it. Allocation is complicated, because we have no input data except for the host itself - so we using any image with appropriate architecture. And to simplify the allocation we skipping defining of the network interfaces for the machine, so that leads to using default VPC. For sure it's possible to find VPC/subnet based on the host zone - that will be a solution for the issue.