aws-samples / aws-lambda-extensions

A collection of sample extensions to help you get started with AWS Lambda Extensions
MIT No Attribution
442 stars 145 forks source link

Lambda Extension for AppConfig - need to support setting internet proxy #68

Closed jxl0319 closed 11 months ago

jxl0319 commented 2 years ago

We have a lambda function which connects to AppConfig but due to security reason lambda function should be inside a VPC. Since there is no VPC endpoint, we will have to route the traffic to on-prem proxy server via transit gateway from where the request will be sent to AppConfig endpoint. In order to use AppConfig, we are using AppConfig extensions layer provided by AWS. Now we would like to know how we can send traffic from AppConfig Extensions to AppConfig via the on-prem proxy server

julianwood commented 2 years ago

HI, This may already be possible. Please try setting the AWS_APPCONFIG_EXTENSION_SERVICE_ENDPOINT Lambda environment variable to the proxy endpoint.

jxl0319 commented 2 years ago

Hi Julian - I've tried setting the AWS_APPCONFIG_EXTENSION_SERVICE_ENDPOINT Lambda environment variable to our on-prem proxy endpoint ( https://xxx.xxx.org:8080 ), also tried the (http://xxx.xxx.org:8080). Both failed. Below are the logs:

Function Logs START RequestId: 85e9977f-7e41-4f94-82cf-747521fc1045 Version: $LATEST [appconfig agent] 2022/03/04 17:19:47 INFO AppConfig Lambda Extension 1.3.0 [appconfig agent] 2022/03/04 17:19:47 INFO serving on port 2772 EXTENSION Name: AppConfigAgent State: Ready Events: [INVOKE,SHUTDOWN] ---- testing code starts ------------- [appconfig agent] 2022/03/04 17:19:52 ERROR could not retrieve /applications/RsvServiceApplication/environments/demoDev/configurations/demoConfig from AppConfig: send request failed (requestId: n/a) [appconfig agent] 2022/03/04 17:19:52 DEBUG error retrieving /applications/RsvServiceApplication/environments/demoDev/configurations/demoConfig (4.307745s) END RequestId: 85e9977f-7e41-4f94-82cf-747521fc1045 REPORT RequestId: 85e9977f-7e41-4f94-82cf-747521fc1045 Duration: 120102.11 ms Billed Duration: 120000 ms Memory Size: 256 MB Max Memory Used: 56 MB Init Duration: 174.48 ms
2022-03-04T17:21:48.002Z 85e9977f-7e41-4f94-82cf-747521fc1045 Task timed out after 120.10 seconds

Questions: We are using west1 gov-cloud. Did you have a similar use case you've tested before? It will be helpful if you can share the details. Currently, our VPC has 3 private subnets, I have the security group with three inbounds to all the subnets at port 443. outbound with all traffic. The only way to get out of internet from this VPC is going through the corporate proxy.
If that environment variable works, what did I miss from my test?

In the environment variables, I also set the following: AWS_APPCONFIG_EXTENSION_HTTP_PORT 2772 AWS_APPCONFIG_EXTENSION_LOG_LEVEL DEBUG AWS_APPCONFIG_EXTENSION_POLL_INTERVAL_SECONDS 45 AWS_APPCONFIG_EXTENSION_POLL_TIMEOUT_MILLIS 3000 AWS_APPCONFIG_EXTENSION_PREFETCH_LIST /applications/RsvServiceApplication/environments/demoDev/configurations/demoConfig

code snipets:

def get_configured_number_of(configuration_type, default): try: print("---- testing code starts -------------") url = f'http://localhost:2772/applications/RsvServiceApplication/environments/demoDev/configurations/demoConfig' req = urlrequest.Request(url) config = json.loads(urlrequest.urlopen(req).read()) print("--- return from AppConfig ---") print(config) if config.get(configuration_type, False): return config.get("numberOfExclamationPoints", default) else: return default except: return default

Thank you for your help! Jian

julianwood commented 2 years ago

Looks like an updated Lambda extension should solve it. Can you update to arn:aws-us-gov:lambda:us-gov-west-1:946746059096:layer:AWS-AppConfig-Extension:20 You may need to modify the IAM permissions for the role you're using for the Lambda function. Previously, appconfig:GetConfiguration was needed, but for the "2.0" version AppConfig switched to the AppConfig Data APIs so you'll need appconfig:StartConfigurationSession and appconfig:GetLatestConfiguration instead.

jxl0319 commented 2 years ago

Hi Julian, I've upgraded to 20, and updated the policies for the lambda IAM role. got a different error as below: [appconfig agent] 2022/03/07 18:45:32 ERROR could not retrieve data for '/applications/RsvServiceApplication/environments/demoDev/configurations/demoConfig': GatewayTimeoutException: Did not receive data before poll timeout

I only set one environment variable: AWS_APPCONFIG_EXTENSION_SERVICE_ENDPOINT = our on-prem proxy

Please advice, Jian

julianwood commented 2 years ago

Thanks, We have an updated AppConfig extension layer we're working on which should resolve this and some other issues. I'll update when this is available.