search

DinoChiesa / Apigee-Java-AWSV4-Signature

Apache License 2.0
2 stars 1 forks source link

Can't get Lambda invocation to work #1

Open guyappy opened 1 year ago

guyappy commented 1 year ago

Hi!

I'm trying to get this to work with a lambda. My setup is:

<AssignMessage name="AM-Construct-Outgoing-AWS-Message">
    <AssignTo createNew="true" type="request">outboundLambdaMessage</AssignTo>
    <Set>
        <Verb>POST</Verb>
        <Headers>
            <Header name="X-Amz-Invocation-Type">RequestResponse</Header>
        </Headers>
        <FormParams/>
        <Payload contentType="application/json">{"name":"foo", "type":"bar"}</Payload>
        <Path>/2015-03-31/functions/arn:aws:lambda:eu-west-1:297922491504:function:bookingExampleFunction/invocations</Path>
    </Set>
</AssignMessage>
<JavaCallout async="false" continueOnError="false" enabled="true" name="JC-AWS-Signature-V4">
    <DisplayName>JC-AWS-Signature-V4</DisplayName>
    <Properties>
        <Property name="debug">true</Property>
        <Property name="service">lambda</Property>
        <Property name="endpoint">https://lambda.eu-west-1.amazonaws.com</Property>
        <Property name="region">eu-west-1</Property>
        <Property name="key">....</Property>
        <Property name="secret">....</Property>
        <Property name="source">outboundLambdaMessage</Property>
    </Properties>
    <ClassName>com.google.apigee.callouts.AWSV4Signature</ClassName>
    <ResourceURL>java://apigee-callout-awsv4sig-20210609.jar</ResourceURL>
</JavaCallout>
<ServiceCallout async="false" continueOnError="false" enabled="true" name="SC-LambdaCallout">
    <DisplayName>SC-LambdaCallout</DisplayName>
    <Request clearPayload="false" variable="outboundLambdaMessage">
        <!--
       No need to set headers now, after signature calculation.
       The following headers were already set by the Java callout:
         x-amz-date, host, authorization, and x-amz-content-sha256
     -->
    </Request>
    <Response>uploadResponse</Response>
    <HTTPTargetConnection>
                <URL>https://lambda.eu-west-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:eu-west-1:297922491504:function:bookingExampleFunction/invocations</URL>
    </HTTPTargetConnection>
</ServiceCallout>

However I'm getting the following error from the serviceCallout 404

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

Page Not Found Page Not Found

Any pointers would be greatly appreciated!

DinoChiesa commented 1 year ago

Hmmm, the configuration you are using looks correct to me. When you say you are getting "404" and "Page Not Found" - where do you see that? Have you used the Apigee trace to see the result of the ServiceCallout ? Is that what is showing you the 404?

Have you used Apigee trace to verify that the Java Callout has succeeded? (With the debug flag for that callout set to true, Apigee should show you extra information in trace variables)

guyappy commented 1 year ago

thanks for getting back to me Dino!

I'm seeing this in the trace indeed: image

The java callout seems to be working fine: image

DinoChiesa commented 1 year ago

Sure thing, Guy. I'm happy to try to help.

The Java callout DOES appear to be working correctly. It has produced a signature, and you can see the value in the variable awsv4_sig_header.Authorization. That variable gets set if and only if the debug property is true. The actual Authorization header gets set on the variable outboundLambdaMessage , but that operation is not shown in Apigee trace because the Authorization header is treated as sensitive. But the same value will be placed in the appropriate header on that message. If you would like some assurance of that, you can interject an AssignMessage between the JavaCallout and the ServiceCallout that will allow you to examine the value via trace. Configure the policy like this: 

<AssignMessage name='AM-Diagnostics'>
  <AssignVariable> 
    <Name>diags</Name> 
    <Ref>outboundLambdaMessage.header.Authorization</Ref>
  </AssignVariable> 
</AssignMessage>

In any case, what appears to be happening is the ServiceCallout is actively receiving a 404 not found from the target you've configured. I wouldn't expect a 404. Normally a service that exists will respond with 403 or 401 if the authorization header is incorrect. Not a 404.

Just as a test I used the command-line tool "curl" to invoke the endpoint, in the same way your ServiceCallout would, and I'm getting a 403 Forbidden, with an appropriate security denial message, telling me the signature is expired. That is what I would expect.

So why would servicecallout be receiving a 404?
You didn't mention which variant of Apigee you are using, or how the networking is configured. Is it possivle that there is something between the Apigee MP and the Amazon target system, that is returning the 404? If you are running Apigee Edge OPDK, or Apigee hybrid, maybe an internal firewall or network gateway might do this. If you are running Apigee X, then maybe your cloud networking configuration might prevent the call and return 404.

To diagnose, see if you can use curl in the same way I tried.... from a workstation that you know has good access to the internet, and then also from the system that runs the message processor.

My curl command is like this: 

curl -i https://lambda.eu-west-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:eu-west-1:297922491504:function:bookingExampleFunction/invocations 
  -H "Authorization: AWS-HMAC-SHA256 Credential=AKIAUKXMQORYKGDIOMW3/20230414/eu-west-1/lambda/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-invocation-type, Signature=0e434522a6c157f7011d10772ef3cb61adc55692d3af22c6982d3ef5701ebca4"
   -H "x-amz-date: 20230414T063223Z" 
   -X POST -d '{"foo": "bar"}'
guyappy commented 1 year ago

hi Dino,

I've tried to run a curl and it provides the same response as Apigee, so that's good to know indeed. I've tried a million things, but one thing that's maybe good to know:

When I change the property of the serviceCallout to:

    <Response>uploadResponse</Response>
    <HTTPTargetConnection>
        <URL>https://lambda.eu-west-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:eu-west-1:297922491504:function:bookingExampleFunction/invocations</URL>
    </HTTPTargetConnection>
</ServiceCallout>

then I get the 404, but the trace also shows the requesturi is:

/2015-03-31/functions/arn:aws:lambda:eu-west-1:297922491504:function:bookingExampleFunction/invocations/2015-03-31/functions/arn:aws:lambda:eu-west-1:297922491504:function:bookingExampleFunction/invocations

which makes sense, this function doesn't exist.

When I change it back to:

    <Response>uploadResponse</Response>
    <HTTPTargetConnection>
        <URL>https://lambda.eu-west-1.amazonaws.com</URL>
    </HTTPTargetConnection>
</ServiceCallout>

I get the following response (also when trying to curl the request: {"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."}

the Authorization header seems to be created fine so I'm really not sure what is up. My full config now is:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<AssignMessage name="AM-Construct-Outgoing-AWS-Message">
    <AssignTo createNew="true" type="request">outboundLambdaMessage</AssignTo>
    <Set>
        <Verb>POST</Verb>
        <Headers>
            <Header name="X-Amz-Invocation-Type">RequestResponse</Header>
        </Headers>
        <FormParams/>
        <Payload contentType="application/json">{"name":"foo", "type":"bar"}</Payload>
        <Path>/2015-03-31/functions/arn:aws:lambda:eu-west-1:297922491504:function:bookingExampleFunction/invocations</Path>
    </Set>
</AssignMessage>
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<JavaCallout async="false" continueOnError="false" enabled="true" name="JC-AWS-Signature-V4">
    <DisplayName>JC-AWS-Signature-V4</DisplayName>
    <Properties>
        <Property name="debug">true</Property>
        <Property name="service">lambda</Property>
        <Property name="endpoint">https://lambda.eu-west-1.amazonaws.com</Property>
        <Property name="region">eu-west-1</Property>
        <Property name="key">....</Property>
        <Property name="secret">...</Property>
        <Property name="source">outboundLambdaMessage</Property>
    </Properties>
    <ClassName>com.google.apigee.callouts.AWSV4Signature</ClassName>
    <ResourceURL>java://apigee-callout-awsv4sig-20210609.jar</ResourceURL>
</JavaCallout>
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ServiceCallout async="false" continueOnError="false" enabled="true" name="SC-LambdaCallout">
    <DisplayName>SC-LambdaCallout</DisplayName>
    <Request clearPayload="false" variable="outboundLambdaMessage">
        <!--
       No need to set headers now, after signature calculation.
       The following headers were already set by the Java callout:
         x-amz-date, host, authorization, and x-amz-content-sha256
     -->
    </Request>
    <Response>uploadResponse</Response>
    <HTTPTargetConnection>
        <URL>https://lambda.eu-west-1.amazonaws.com</URL>
    </HTTPTargetConnection>
</ServiceCallout>
DinoChiesa commented 1 year ago

oh! You're getting closer. You have seen that the Path you specify in the AssignMessage, is appended to the Path you specify in the HTTPTargetConnection/URL field. So you corrected that, good.

OK that leaves the signature match problem. Solving that kind of problem can be frustrating, but it's usually something basic. You're certain of the key I suppose. The signing method is the tricky part.

For your lambda to accept inbound POST messages, what is the policy? Must the payload be signed? I am not an expert on AWS lambda so am not certain what the default behavior is. By default the callout does not include the sha256 of the payload (POST body) in the list of signed headers. And in your case I can see that the callout is signing these headers:

and it is not including a header like x-amz-content-sha256 in the request, which inevitably means that header is not included in the signed base.

To ask that the callout include the SHA256 of the payload in the list of signed headers, you need to set the property sign-content-sha256 to true in the JavaCallout policy. This should be easy to try.

guyappy commented 1 year ago

thanks for all the help Dino!

I tried your suggestion and a lot of other things but I can't get it to work...

Calling the lambda does work for example through Postman where one can configure the AWS authorization settings in th app itself, so not sure what's going on in Apigee.

yilo90 commented 2 weeks ago

Hello, in case somebody else is struggling: I had the same issue with {"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."}.

I don't know if that is the issue, but the value of the canonical request (awsv4sig_creq) was POST↵/2015-03-31/functions/arn%3aaws%3alambda%3aeu-central-1%3...

I suspect, that there is a problem with special character like the semicolon (arn:aws:lambda...) which is encoded to %3a. Thats why I made use of lambda function URL. To use the lambda function URL you have to empty the "Path" attribute in the Construct-Message policy and to update the endpoint and URL in the other policies.

How to invoke a lambda function url with AWS signature: https://www.youtube.com/watch?v=MXXq1M9gYY0

Hope that helps anybody.