Deploy lambda to VPC. - Githubissues

jarrettj commented 6 years ago

Do you want to request a feature or report a bug? Feature

What is the current behavior? There's no option to deploy a lambda into a VPC.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem. When using the current function add functionality you are not promoted to place the function into a VPC.

What is the expected behavior? Would be helpful to deploy to a VPC.

Which versions of Amplify CLI, and which OS are affected by this issue? Did this work in previous versions? 0.1.14

abualsamid commented 5 years ago

any update/timeline on this? and/or is there a workaround, manual or otherwise, to let us deploy our lambda's to a vpc.

paultipper commented 5 years ago

Any progress on this issue? I've found that if I manually reconfigure the lambda to deploy to my default VPC in the AWS Console, the next time I run the amplify publish command, the VPC setting is reset back to "No VPC". This is a deal killer for me as far as being able to use amplify to build full-stack apps, which is heart-breaking, because otherwise it's awesome.

BabyDino commented 5 years ago

I was also looking into this. I found a (temp?) solution:

Change your <functionname>-cloudformation-template.json:

Under Resources add:

"VpcConfig": {
   "SecurityGroupIds": [
       "sg-xxx"
   ],
   "SubnetIds": [
       "subnet-xxx",
       "subnet-xxx",
       "subnet-xxx"
   ]
}

and add a statement to your execution role:

{
   "Effect": "Allow",
   "Action": [
       "ec2:CreateNetworkInterface",
       "ec2:DescribeNetworkInterfaces",
       "ec2:DeleteNetworkInterface"
   ],
   "Resource": "*"
}

Refer to: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-function.html

paultipper commented 5 years ago

@BabyDino Hi Stefan, thanks for the tip. Whereabouts exactly did you add the execution role statement in your -cloudformation-template.json document? I tried adding it to Resources/lambaexecutionpolicy/Properties/PolicyDocument/Statement, but when I tried to deploy using the amplify publish command, I got the following error:

`Following resources failed

Resource Name: cdkamplifyappfe31e78f (AWS::Lambda::Function) Event Type: update Reason: The provided execution role does not have permissions to call CreateNetworkInterface on EC2 (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: 0db4f5a3-1fc7-428f-96fd-3d89f464ffa3) `

I'm guessing I added the execution role statement in the wrong position in the -cloudformation-template.json file.

BabyDino commented 5 years ago

@paultipper this works for us:

"lambdaexecutionpolicy": {
    "DependsOn": [
        "LambdaExecutionRole"
    ],
    "Type": "AWS::IAM::Policy",
    "Properties": {
        "PolicyName": "lambda-execution-policy",
        "Roles": [
            {
                "Ref": "LambdaExecutionRole"
            }
        ],
        "PolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Action": [
                        "logs:CreateLogGroup",
                        "logs:CreateLogStream",
                        "logs:PutLogEvents"
                    ],
                    "Resource": {
                        "Fn::Sub": [
                            "arn:aws:logs:${region}:${account}:log-group:/aws/lambda/${lambda}:log-stream:*",
                            {
                                "region": {
                                    "Ref": "AWS::Region"
                                },
                                "account": {
                                    "Ref": "AWS::AccountId"
                                },
                                "lambda": {
                                    "Ref": "LambdaFunction"
                                }
                            }
                        ]
                    }
                },
                {
                    "Effect": "Allow",
                    "Action": [
                        "ec2:CreateNetworkInterface",
                        "ec2:DescribeNetworkInterfaces",
                        "ec2:DetachNetworkInterface",
                        "ec2:DeleteNetworkInterface"
                    ],
                    "Resource": "*"
                }
            ]
        }
    }
}

paultipper commented 5 years ago

@BabyDino Yeah, that's exactly where I put the new statement, but I'm still getting the error. Stumped. :(

BabyDino commented 5 years ago

@paultipper I do not have an answer sorry, just figured this out myself. There are some articles about your error though: https://medium.com/@onclouds/aws-codestar-lambda-vpc-706fcf1252d4.

Hope this helps.

paultipper commented 5 years ago

@BabyDino Finally cracked it! You have to add and deploy the network interface permissions statement first before you'll be allowed to add the VPC configuration statement. Once the network interface permissions are in place, then you'll be able to add the VPC config, but you can't add the VPC and network interface permissions statements at the same time.

Kudos to gozup for pointing this out - see this thread.

And many thanks, Stefan, for all your help - you got me most of the way there!

BabyDino commented 5 years ago

@paultipper You're welcome, glad it works!

Thank you for the link, I was not aware of that. I'll might consider creating a role in IAM and just use that role instead of creating a role with every function. Most of our functions require the same permissions anyway.

Or something with DependsOn. I'm fairly new to CF.

regischow commented 4 years ago

@BabyDino another way to do it would be to add arn:aws:iam::aws:policy/service- role/AWSLambdaVPCAccessExecutionRole as ManagedPolicyArns to the LambdaExecutionRole.

anaji commented 4 years ago

it works after adding the vpc config inside resources properties in -cloudformation-template.jsonc file . i.e:

`"Resources": {
    "LambdaFunction": {
        "Type": "AWS::Lambda::Function",
        "Metadata": {
            "aws:asset:path": "./src",
            "aws:asset:property": "Code"
        },
        "Properties": {
            "Handler": "index.handler",
              ....
            "VpcConfig": {
                "SecurityGroupIds": [
                    "sg-xxx"
                ],
                "SubnetIds": [
                    "subnet-xxx",
                    "subnet-xxx",
                    "subnet-xxx"
                ]
            }
        },
        "LambdaExecutionRole": {
              ....
        `

batical commented 4 years ago

and how to deal with multi env and lambda vpc except doing some manual script to update template json

alexkates commented 4 years ago

and how to deal with multi env and lambda vpc except doing some manual script to update template json

This is something I'm wrapping my head around also. My current thought is to add a network category via https://aws-amplify.github.io/docs/cli-toolchain/quickstart#custom-cloudformation-stacks, and provision a VPC, Subnets, Security Groups, Route Tables, IGWs, etc. This will give me those resources for each environment, and allow me to link my functions to that VPC via Refs.

I was hoping that someone from the amplify-cli team could offer their thoughts about this as a viable workaround?

lorengordon commented 4 years ago

Just had to do this ourselves... To deal with the dependency on the IAM role needing the EC2 Network Interface permissions before the role is attached to the lambda, we used a separate inline policy and a DependsOn block... This allowed us to keep the restrictive policy that amplify creates for CloudWatch Logs, rather than use AWSLambdaVPCAccessExecutionRole (which has no resource restriction on which log groups/streams the lambda can write to). Here's a diff:

@@ -23,6 +23,9 @@
        },
        "Resources": {
                "LambdaFunction": {
+                       "DependsOn": [
+                               "LambdaExecutionPolicyCustom"
+                       ],
                        "Type": "AWS::Lambda::Function",
                        "Metadata": {
                                "aws:asset:path": "./src",
@@ -48,6 +51,19 @@
                                                }
                                        ]
                                },
+                               "VpcConfig": {
+                                       "SecurityGroupIds": [
+                                               "sg-xxxxxx"
+                                       ],
+                                       "SubnetIds": [
+                                               "subnet-xxxxxx",
+                                               "subnet-xxxxxx",
+                                               "subnet-xxxxxx",
+                                               "subnet-xxxxxx",
+                                               "subnet-xxxxxx",
+                                               "subnet-xxxxxx"
+                                       ]
+                               },
                                "Environment": {
                                        "Variables": {
                                                "ENV": {
@@ -154,6 +170,104 @@
                                        ]
                                }
                        }
+               },
+               "LambdaExecutionPolicyCustom": {
+                       "Type": "AWS::IAM::Policy",
+                       "Properties": {
+                               "PolicyName": "lambda-execution-policy-custom",
+                               "Roles": [
+                                       {
+                                               "Ref": "LambdaExecutionRole"
+                                       }
+                               ],
+                               "PolicyDocument": {
+                                       "Version": "2012-10-17",
+                                       "Statement": [
+                                               {
+                                                       "Effect": "Allow",
+                                                       "Action": [
+                                                               "ec2:CreateNetworkInterface",
+                                                               "ec2:DescribeNetworkInterfaces",
+                                                               "ec2:DeleteNetworkInterface"
+                                                       ],
+                                                       "Resource": "*"
+                                               }
+                                       ]
+                               }
+                       }

semirenko commented 3 years ago

here is the way how I did env specific configuration:

  "Conditions": {
.........
    "CurrentEnvIsLive": {
      "Fn::Equals": [
        {
          "Ref": "env"
        },
        "live"
      ]
    }
  },
.......
        "VpcConfig": {
          "Fn::If": [
            "CurrentEnvIsLive",
            {
              "SecurityGroupIds": [
                "sg-xxxxx"
              ],
              "SubnetIds": [
                "subnet-xxxx"
              ]
            },
            {
              "SecurityGroupIds": [
                "sg-yyyyyy"
              ],
              "SubnetIds": [
                "subnet-yyyy"
              ]
            }
          ]
        },

corydorning53 commented 3 years ago

here is the way how I did env specific configuration:

  "Conditions": {
.........
    "CurrentEnvIsLive": {
      "Fn::Equals": [
        {
          "Ref": "env"
        },
        "live"
      ]
    }
  },
.......
        "VpcConfig": {
          "Fn::If": [
            "CurrentEnvIsLive",
            {
              "SecurityGroupIds": [
                "sg-xxxxx"
              ],
              "SubnetIds": [
                "subnet-xxxx"
              ]
            },
            {
              "SecurityGroupIds": [
                "sg-yyyyyy"
              ],
              "SubnetIds": [
                "subnet-yyyy"
              ]
            }
          ]
        },

@semirenko do you mind posting the files your changed and where the changes were made to handle multi-environment? also, where you define the environment...

semirenko commented 3 years ago

@corydorning53 , env comes as Lambda function parameter in XXXX-cloudformation-template.json file.

  "Parameters": {
    "CHALLENGEANSWER": {
      "Type": "String",
      "Default": ""
    },
    "modules": {
      "Type": "String",
      "Default": "",
      "Description": "Comma-delimmited list of modules to be executed by a lambda trigger. Sent to resource as an env variable."
    },
    "resourceName": {
      "Type": "String",
      "Default": ""
    },
    "trigger": {
      "Type": "String",
      "Default": "true"
    },
    "functionName": {
      "Type": "String",
      "Default": ""
    },
    "roleName": {
      "Type": "String",
      "Default": ""
    },
    "parentResource": {
      "Type": "String",
      "Default": ""
    },
    "parentStack": {
      "Type": "String",
      "Default": ""
    },
    "env": {
      "Type": "String"
    }
  },

It was added by amplify CLI, as part of lambda files generation result. In my case this is a cognito trigger, part of Auth category. Not sure, if Amplify adds it also in case of regular Lambda function.

I just added CurrentEnvIsLive section into Conditions block of the same file, which is a standard part of any CloudFormation file. Same for VpcConfig. It is also a part of CF file specs, Resources -> LambdaFunction -> Properties -> VpcConfig https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-lambda-function-vpcconfig.html

ps2goat commented 3 years ago

I was stepping through each of the ec2 permissions to try and limit what the lambda can do, but got stuck on the DeleteNetworkInterfaces permission-- Apparently you can't limit it to a region, it requires *. And if I can't limit that permission, I gave up on the rest. (Permission checks start with CreateNetworkInterfaces -> DescribeNetworkInterfaces -> DeleteNetworkInterface, and I stopped there because of this.)

I'm more concerned with exactly which interfaces it can access and delete more than the logs it can generate. I'm sure Amazon limits this, but it'd be nice to say "create interfaces under some identifier, and you can only attach, detach, create, or delete with those identifers/buckets"

lorengordon commented 3 years ago

@ps2goat ec2:DeleteNetworkInterface does support tag conditions with ec2:ResourceTag/${TagKey}. So presumably you could tag the network interfaces, and use a condition in the policy to allow deleting network interfaces only if they have a matching tag.

caiodiletta commented 2 years ago

This works, but how would I do that if I have different vpc for different envs?

vishalrajole commented 2 years ago

Any update when this will be supported? Thanks

bothra90 commented 2 years ago

@caiodiletta: You can add the env-specific VPC configuration to the team-provider-info.json file as:

{
  "<envName>": {
    ...
    "categories": {
      ...
      "function": {
        "<functionName>": {
          "subnetIds": [
            "subnet-xxxxxxxxxxxxx"
          ],
          "securityGroupIds": [
            "sg-xxxxxxxxxxxxxxxxxxx",
            "sg-xxxxxxxxxxxxxxxxxxx"
          ],
          ...
        }
      }
    }
  }
}

This can then be taken as parameters by the cloud formation template and referred in the function config:

{
  ...
  "Parameters": {
    ...
    "subnetIds": {
      "Type": "CommaDelimitedList"
    },
    "securityGroupIds": {
      "Type": "CommaDelimitedList"
    }
  },
  "Resources": {
    "LambdaFunction": {
      "Properties": {
        "VpcConfig": {
          "SecuritGroupIds": {
            "Ref": "securityGroupIds"
          },
          "SubnetIds": {
            "Ref": "subnetIds"
          }
        }
      }
    },
    ...
  }
}

koren-tako-storrsoft commented 2 years ago

You could perform the VpcConfig configuration by using CloudFormation external values and then: "VpcConfig": { "SecurityGroupIds": [ { "Fn::ImportValue": { "Fn::Sub": [ "${ENV}-VPCSecurityGroup", { "ENV": { "Ref": "env" } } ] } } ], "SubnetIds": { "Fn::Split": [ ",", { "Fn::ImportValue": { "Fn::Sub": [ "${ENV}-VPCPrivateSubnets", { "ENV": { "Ref": "env" } } ] } } ] } }

eettaa commented 2 years ago

+1 for official guidance here. In particular, it's my understanding that this is needed in order to have a static outbound IP address for a lambda function (required if your 3p services have ip allowlist ranges). This issue, specifically "I've found that if I manually reconfigure the lambda to deploy to my default VPC in the AWS Console, the next time I run the amplify publish command, the VPC setting is reset back to "No VPC".", seems to directly contradict the official AWS docs on static outbound IP's for lambdas