okigan / awscurl

curl-like access to AWS resources with AWS Signature Version 4 request signing.
MIT License
737 stars 91 forks source link

Check this if you run into issues with execute-api WebSockets #61

Open atlascoder opened 5 years ago

atlascoder commented 5 years ago

I got error when trying to execute:

awscurl --service execute-api -X POST -d "hello" https://spjhtm1qcd.execute-api.us-east-1.amazonaws.com/test/@connections/atntqfjloAMCECg=

Also tried:

awscurl --service execute-api -X POST -d "hello" https://spjhtm1qcd.execute-api.us-east-1.amazonaws.com/test/\@connections/atntqfjloAMCECg\=

and

awscurl --service execute-api -X POST -d "hello" "https://spjhtm1qcd.execute-api.us-east-1.amazonaws.com/test/@connections/atntqfjloAMCECg="

The error:

{"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.\n\nThe Canonical String for this request should have been\n'POST\n/test/%40connections/atntqfjloAMCECg%3D\n\nhost:spjhtm1qcd.execute-api.us-east-1.amazonaws.com\nx-amz-date:20190603T171519Z\n\nhost;x-amz-date\n2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824'\n\nThe String-to-Sign should have been\n'AWS4-HMAC-SHA256\n20190603T171519Z\n20190603/us-east-1/execute-api/aws4_request\nb013b0bc1c19a4ebc9da8d82a6122379899465e0cd4808187a4ca276e999c56a'\n"}
Traceback (most recent call last):
  File "/Users/anton.titkov/Library/Python/2.7/bin/awscurl", line 11, in <module>
    sys.exit(main())
  File "/Users/anton.titkov/Library/Python/2.7/lib/python/site-packages/awscurl/awscurl.py", line 373, in main
    r.raise_for_status()
  File "/Users/anton.titkov/Library/Python/2.7/lib/python/site-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://spjhtm1qcd.execute-api.us-east-1.amazonaws.com/test/@connections/atntqfjloAMCECg=
okigan commented 5 years ago

@atlascoder your url does not look correct '@connections' seems wrong (looks similar to something used in aws cloudformation, but not a correct api gateway path)

atlascoder commented 5 years ago

Hi @okigan , actually this is correct url for WebSocket API of APIGateway: https://docs.aws.amazon.com/en_us/apigateway/latest/developerguide/apigateway-how-to-call-websocket-api-connections.html

okigan commented 5 years ago

Oh this is something new (to me :), will require more digging

atlascoder commented 5 years ago

Yes, your tool is convenient and support for WebSocket url would be great

okigan commented 5 years ago

Ooops, I did not realize my tool made into into official AWS docs! 🤣 https://docs.aws.amazon.com/en_us/apigateway/latest/developerguide/apigateway-how-to-call-websocket-api-connections.html

okigan commented 5 years ago

@atlascoder ok, this is getting interesting, I need more information to reproduce this.

Paste your setup for the route, and how you’ve setup the request and response template/mapping/etc:

https://docs.google.com/document/d/1-5u7A-bqmYldUoTBy6rMsO4rJC_kQzaHjHs__1yfmHY

atlascoder commented 5 years ago

@okigan, sorry, it's not so straight.. I create WebSocket connection from Qt app and then I take session ID and try to send data to the socket with POST request. To reproduce this you can create any WebSocket-type API and a resource that does not require authorization. The resource should have Lambdas connected to $connect, $disconnect and default. Use any convenient language that have WebSocket lib in order to make connection. In $connect lambda you will find connectionId in requestContext.

If I understood right the aim of your tool - it can make requests using AWSCredentials from env, so, if a current user will be authorized to post data to websocket - the POST requests like https://api_endpoint/stage/@connection/connectionId should send content to the connection (while it is open)

okigan commented 5 years ago

It’s more about that back end setup (not the qt setup), I tried to follow https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-how-to-call-websocket-api-wscat.html. But effectively it returns an error that request/response templates are not setup.

So I need more insight how to setup the WebSocket api (that works👻)

okigan commented 5 years ago

@atlascoder I think i have a repro (instead of mocking created a simple lambda), try out this branch confirm that it works for you as well: https://github.com/okigan/awscurl/pull/64

okigan commented 5 years ago

ok, i've figured it out, fantastically complicated, strap on your seatbelts! This will require usage of both tools: wscat and awscurl.

  1. connect to the endpoint using wscat
  2. send something to get connection information
  3. copy the connectionId to the end of awscurl command
  4. post some data to the connection URL
  5. get some info about the connection/websocket
  6. delete the connection/websocket
  7. PROFIT!!!
awscurl with ws

code for the connected lambda function:

exports.handler = (event, context, callback) => {
    console.log("event:", event)
    console.log("event:", event)

    var result = {
        // event, 
        // context,
        "connectionId": event.requestContext.connectionId,
        "for": "awscurl"
    };

    var response = {
        "statusCode": 200,
        "headers": {
            "Content-Type": "*/*"
        },
        body: JSON.stringify(result)
    }

    console.log("response:", response)

    callback(null, response);
};

IMG_4118

atlascoder commented 5 years ago

@okigan cool! So, everything is needed is to make percent-encoding... Obvious! Thank you!

atlascoder commented 5 years ago

I'll check the branch #64 and reply

atlascoder commented 5 years ago

@okigan did you use WS API without authorization? Or how have you used wscat with AWS credentials?

okigan commented 5 years ago

@atlascoder without, see updated snapshot above

atlascoder commented 5 years ago

OK, @okigan thank you!

atlascoder commented 5 years ago

@okigan, I confirm - it works! Thanks a lot!!

subratamazumder commented 4 years ago

@okigan thanks a lot 💯 , I was really struggling to get execute-api working with web-socket. Your clear notes with screen shots made it clear as water 👍

okigan commented 4 years ago

I've made a proposal to AWS to update documentation, all encouraged to review & comment on the draft.

current: https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-how-to-call-websocket-api.html

proposal draft: https://docs.google.com/document/d/1G_vyFBV7d_3QS4dcaRqexLa8magZm7CsPAuivGTRip8/edit?usp=sharing

okigan commented 4 years ago

hmm, might be simpler to make a PR for https://github.com/awsdocs/amazon-api-gateway-developer-guide/blob/master/doc_source/apigateway-how-to-call-websocket-api-connections.md

jamessouth commented 3 years ago

This worked for me, using the https://github.com/legal90/awscurl tool. I have not been able to make messaging clients work with the Go SDK, but this shows the APIGW works.