nicholasjackson
commented
4 years ago Hi @CHCP, there are a couple of solutions to do this. In all the options below I have assumed that you are interested in integrating non lambda based applications and that lambda to lambda calls would use standard ARN routing and security policy.
One of the core problems is the Lambda execution model. Lambda is designed to scale as the load on your system increases. All the facets of Lambda such as the recommended data stores you use with it are designed to work with this horizontal scale. In the instance that Lambda needs to connect to a system which is outside of this model the user must handle these requirements. To participate in a Service Mesh a Lambda function needs an identity which is in the form of an x509 certificate issued by the Consul server. While Lambda functions do not immediately scale to 0 after a request has finished the lifecycle is not controllable by the user. Consideration needs to be given that peaks in load on the system will cause new Lambda instances to be created, this will increase the load on the Server signing the identity requests.
Custom runtime It is possible to create a custom runtime in Lambda, you could bundle Envoy as a side process which intercepts ingress and egress from the function. Envoy would connect to Consul in the standard way to obtain endpoints and identity.
https://github.com/awslabs/aws-lambda-container-image-converter
Advantages: All of the features L7 and security can be leveraged thanks to Envoy proxy Lambda retains single identity and access can be controlled through service mesh authentication (mTLS) and authorisation (Intentions)
Disadvantages: Slower function start times Potential load on Consul in the instance you have many functions which start simultaneously. For every function instance started Consul needs to sign certificates for mTLS Additional complexity of over standard lambda approach
Go SDK It is possible to use the Go SDK inside of AWS lambda to validate ingress and also provide egress using upstream model. I have done a couple of experiments with this approach, some old code can be found in the repo below.
(https://www.consul.io/docs/connect/native.html)[https://www.consul.io/docs/connect/native.html]
https://github.com/nicholasjackson/consul-connect-router/blob/master/lambda/main.go
Advantages: Lambda retains single identity and access can be controlled through service mesh authentication (mTLS) and authorisation (Intentions) Go SDK has very low latency
Disadvantages: L4 only from the SDK, L7 capabilities such as routing and reliability need to be manually built SDK is currently only available for GoLang, Consuls API does support all the features needed to obtain certificates and discovery, but again there would need to be engineering effort to make this work Slower function start times Potential load on Consul in the instance you have many functions which start simultaneously. For every function instance started Consul needs to sign certificates for mTLS Complexity of over standard lambda approach
Ingress / Egress Gateway It would be possible to de-couple the sidecar process and run Envoy in a VM autoscale group which would be used to proxy traffic to and from AWS lambda. This approach would require a trusted network zone between AWS lambda and the proxy VMs running the proxy.
https://aws.amazon.com/blogs/compute/announcing-improved-vpc-networking-for-aws-lambda-functions/
Advantages: Decoupled proxy from lambda invocation gives predicable load on Consul Proxy is only forwarding network traffic and should be capable of multiplexing many lambda connections Standard autoscale mechanisms can be employed to control the gateway group size
Disadvantages: Complexity of over standard lambda approach Potentially reduced security as a Gateway approach treats all lambda functions as a single identity Additional cost for running gateway VMs
rts-gordon
schristoff
Feature Description
Use Consul Connect in AWS Lambda.
Use Case(s)
We have launched a lots of mircoservices in AWS Lambda, we want to add Service Discovery and Service Registry feature in Lambda. Can we use Consul Connect in AWS Lambda, and would you like to give me some examples. Thanks a lot.