`from_tags` should work with managed VPC subnets

[Lou1415926]

What is the bug

Suppose I let Copilot created managed VPC (as opposed to imported), with two public subnets and two private subnets. I tagged a public subnet with "type: public", and a private subnet with tag "type: private".

If my service has this manifest below:

network:
  vpc:
    placement:
      subnets:
        from_tags:
          type: "private"

Copilot won't automatically create a NAT.

If my service has this manifest instead:

network:
  vpc:
    placement:
      subnets:
        from_tags:
          type: "public"

Copilot will keep "AssignPublicIp" disabled.

A result of both cases is that my service is not able to pull image from ECR.

What is expected

Since the VPC is managed, even though they are introduced into the service via. from_tags, Copilot should be able to tell whether they are private/public and create NAT/enable public IP accordingly. The behavior should be consistent with placement.type.

Related: Gitter

[WilburJZHAO]

How to work around this issue for now? I am trying to use a specific public subnet as placement. My service is not able to pull image from ECR.

[huanjani]

Hi, @WilburJZHAO!

Apologies for the delayed response-- the Copilot team was exploring and discussing your question.

Is the subnet you'd like to use in your environment?

network:
  vpc:
    subnets:
      public: 

If so, AssignPublicIP should be enabled.

Also, is there a reason you don't want to just use the default setting, placing the service in public subnets? Is it because you'd like to use the subnets you already have? As an aside, you should have two subnets for Load Balanced Web Services.

With a little more information, hopefully we can help you get deployed asap!

Thanks and again, sorry for the delay.

[WilburJZHAO]

Hi @huanjani,

Thanks for your reply.

What I did is like this:

network: vpc: placement: subnets: [public-subnet-id]

And the reason why I want to use only one subnet as the placement is because it is in the same availability zone with my RDS database, which provides the best performance. The service on different availability zones with my RDS database will be 3 time slower than the one in the same availability zone with my RDS database.

[huanjani]

Thanks! Is that subnet that you're specifying one that you have imported into your Copilot environment?

[WilburJZHAO]

I tried import subnets from existing and create subnets by Copilot. Both have the same error.

[WilburJZHAO]

I can see from your codebase in /internal/pkg/deploy/cloudformation/stack/transformers.go, line 906-944, in convertNetworkConfig function. If a user set network.VPC.Placement, the AssignPublicIP option will be disabled.

[huanjani]

Okay, @WilburJZHAO-- thank you so much for your patience as we figure out how to best deal with this bug. Thanks for handling our confirmation questions.

We are working to fix this bug and hope to have it out in our next release early next week!

Again, apologies for the churn and back-and-forth. I know you were also working with a Cloud Support Engineer on this.

[huanjani]

Hi, @WilburJZHAO. In our next release, we will be introducing an overrides feature that will unblock you while we work toward a longer-term solution: https://github.com/aws/copilot-cli/issues/4208.

With that, you will be able to:

1. run copilot svc override and select yamlpatch, which will cause Copilot to generate a copilot/backend/override/cfn.patches.yml file.
2. edit the created cfn.patches.yml file with:

   - op: replace
   path: /Resources/Service/Properties/NetworkConfiguration/AwsvpcConfiguration/AssignPublicIp
   value: "ENABLED"

3. run copilot svc deploy

Stay tuned, and let us know how it goes once that feature is available! Thanks again for your patience.

[WilburJZHAO]

Thanks @huanjani

[WilburJZHAO]

Hi @huanjani ,

After using the updated version 1.27.0, I cannot update my service successfully. With the same manifest.yml (use default subnets settings) file, if I returned back to version 1.26.0, I can update the service.

Here is my http in manifest.yml:

And in the ListenerRule, all http requests are Redirect to https.

Did I miss something?

[paragbhingre]

@WilburJZHAO could you please paste in your service manifest here? And could you please also explain what were you trying to do and how did you end up in this error?

[WilburJZHAO]

Hi @paragbhingre,

Here is my service manifest:

# Read the full specification for the "Load Balanced Web Service" type at:
#  https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/

# Your service name will be used in naming your resources like log groups, ECS services, etc.
name: backend
type: Load Balanced Web Service

# Distribute traffic to your service.
http:
  # Requests to this path will be forwarded to your service.
  # To match all requests you can use the "/" path.
  path: '/'
  # You can specify a custom health check path. The default is "/".
  healthcheck: '/aws-health-check'

# Configuration for your containers and service.
image:
  # Docker build arguments. For additional overrides: https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/#image-build
  build: ../Dockerfile
  # Port exposed through your container to route traffic to it.
  port: 9001

cpu: 1024       # Number of CPU units for the task.
memory: 2048    # Amount of memory in MiB used by the task.
count:
  range: 1-3
  cooldown:
    in: 180s
    out: 60s
  cpu_percentage: 70
  response_time: 10s
exec: true     # Enable running commands in your container.
network:
  connect: true # Enable Service Connect for intra-environment traffic between services.

I updated the package to the latest version 1.27.0 this morning.

I were trying to update my service, so I run copilot svc deploy and the error came, which was not there when I use version 1.26.0. I do have a listener rule for http but all http requests are redirect to https. And https listener rule will forward requests to the service.

I downgrade the package to 1.26.0, run again copilot svc deploy with the same manifest file. Worked fine.

[paragbhingre]

Thank you for providing the manifest details to us. I tried following the same steps as you did, and I could successfully deploy the same service from v1.26.0 to v1.27.0. To dig deeper into your issue, could you please also provide us with the following information -

1. Have you ever manually changed the listener rule created by Copilot?
2. What do you mean by http requests are redirected to https? In your case, we do not expect to have any redirection to HTTPS at all. Could you please paste the screenshot of your listener rule showing how it is redirecting the traffic to HTTPS?

[WilburJZHAO]

@paragbhingre Thanks for your reply.

1. Yes, I manually changed the listener rule to meet the requirements
2. You can see the screenshot below. How can I achieve this by using manifest file if I don't manually change listener rules? I have two services, different host header will point to different service.

[paragbhingre]

@WilburJZHAO If I am understanding it correctly, then these screenshots are after you modified the listener rules manually. And these screen shots are what you want to achieve with Copilot without manually changing the listener rules. Am I correct?

[WilburJZHAO]

@paragbhingre Yes, you are correct.

[paragbhingre]

Oh, that should be easy with Copilot. Let us consider that you have serviceA and serviceB. Let's say we want serviceA to be reachable on "v1.example.com" and serviceB on "v2.eample.com". In this case, for serviceA we will set http.alias field as "v1.example.com". And for serviceB we will set http.alias field as "v2.example.com". This way, Copilot will create 2 listeners rules for serviceA, i.e., 1. An HTTP listener rule redirecting traffic to HTTPS listener and 2. An HTTPS listener rule will route this traffic to the desired target group. Similar things will happen with ServiceB as well. So here is your manifest for serviceA with an http.alias field added to it, which will do the needful. 

# Read the full specification for the "Load Balanced Web Service" type at:
#  https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/

# Your service name will be used in naming your resources like log groups, ECS services, etc.
name: backend
type: Load Balanced Web Service

# Distribute traffic to your service.
http:
  # Requests to this path will be forwarded to your service.
  # To match all requests you can use the "/" path.
  path: '/'
  alias: "v1.example.com" 
  # You can specify a custom health check path. The default is "/".
  healthcheck: '/aws-health-check'

# Configuration for your containers and service.
image:
  # Docker build arguments. For additional overrides: https://aws.github.io/copilot-cli/docs/manifest/lb-web-service/#image-build
  build: ../Dockerfile
  # Port exposed through your container to route traffic to it.
  port: 9001

cpu: 1024       # Number of CPU units for the task.
memory: 2048    # Amount of memory in MiB used by the task.
count:
  range: 1-3
  cooldown:
    in: 180s
    out: 60s
  cpu_percentage: 70
  response_time: 10s
exec: true     # Enable running commands in your container.
network:
  connect: true # Enable Service Connect for intra-environment traffic between services.

So Copilot takes care of creating the certificates for the HTTPS listener rules if your app is created with --domain.

Please let us know how that goes. Here to help.

[WilburJZHAO]

@paragbhingre Thanks for your help. I can understand what you suggested and I think it will work perfectly if create a new environment. But I am still facing the issue One or more listeners not found when I was trying to updated my current services. I think this is because I manually changed the listener rules. The system cannot find the rules originally created by the copilot. I don't want to totally delete my environment and services and start from the beginning. How can I solve this? How can I force the system to create new listener rules like I first time deploy? I tried copilot svc deploy --force with no luck.

[paragbhingre]

@WilburJZHAO I understand what you are saying. What I think is that if you can delete your services and then try deploying them again, that will solve your problem, and you also don't need to delete the environment in that case. But I am not sure if you would want to delete your services at all. So when you delete your services, Copilot will remove the load balancers from the environment. When you deploy your first service in the environment, that is when Copilot creates a load balancer with listeners and listener rules with it. For now, I can only think of this solution, Do let us know if this solution works fine with you, and in the meantime, I will try to find if there is another way we can get around this.

Edit - Another solution that you can try is to change the path of serviceA to something other than '/' and deploy the service. Similarly, change the path for serviceB  and deploy that service as well. Once both services are deployed, reset the path of serviceA to '/' and do the deployment. Similarly, change the path of the serviceB to '/' and deploy that service as well, and see if that works. 

[paragbhingre]

Another solution that you can try is to change the path of serviceA to something other than '/' and deploy the service. Similarly, change the path for serviceB and deploy that service as well. Once both services are deployed, reset the path of serviceA to '/' and do the deployment. Similarly, change the path of the serviceB to '/' and deploy that service as well, and see if that works.

You can also take a look at this issue which seems pretty similar to yours.