search

aws-solutions / aws-data-lake-solution

A deployable reference implementation intended to address pain points around conceptualizing data lake architectures that automatically configures the core AWS services necessary to easily tag, search, share, and govern specific subsets of data across a business or with other external businesses.
https://aws.amazon.com/solutions/implementations/data-lake-solution/
Apache License 2.0
401 stars 160 forks source link

Signing API Request to application #46

Closed skirk-mpr closed 4 years ago

skirk-mpr commented 4 years ago

I'm having trouble getting the API access to the application working. I'm getting Unauthorized responses, so I'm assuming I'm missing something with respect to the signing process for the requests. I am using the provided JS code to generate this signature for my request, as well as, implemented the signing functionality in another language - which I'm getting the same hash output as the provided JS function, for the same input data. Could you clarify the some of the specifics regarding the steps to sign the requests for this application? (Reference: http://docs.awssolutionsbuilder.com/data-lake/api/working-with-api/)

  1. per the documentation, the endpoint parameter to the signing function should not include the leading "https://" or end in a slash ("/") ?
  2. the example apiEnpoint has "Prod" in the URL cased with a leading capital 'P' -- however, when I inspect the network traffic via Developer tools, the Request URL indicates this is lowercase "prod" for our implementation -- should we use the casing that matches what we see in our deployment?
  3. the apiEndpoint parameter is always just the same value for your specific instance of the application, regardless of what specific endpoint you are signing a request for -- e.g POST to /packages/new vs. a GET to /cart, would both take the same apiEndpoint value as input to the signing function?
  4. The strings "DATALAKE4", "datalake" and "datalake4_request" that are included in the provided function to create the keys for the hashing or used as values to actually hash the data for various steps remain constant for all requests to the application, right? The only application deployment instance specific information is the accesKey, secretKey, dateStamp and apiEndpoint?

Lastly, I noticed that traffic from the the web application uses Cognito authernatication with a JWT in the "Auth" header - "tk:" instead of "ak". Is an acceptable access pattern for integrating with the API to hook into Cognito with a custom service user that can retireve its own JWT before running API calls against the API with this token instead of the signing process?

Thank you!

beomseoklee commented 4 years ago

@skirk-mpr I hope this would answer your questions.

  1. You can just put endpoint without https:// and /. For example, HASH.execute-api.AWS_REGION.amazonaws.com.
  2. It's prod.
  3. Yes, it's same. Just you can use HASH.execute-api.AWS_REGION.amazonaws.com.
  4. I realized you don't need dateStamp since that parameter is not used in the example function in the documentation. For your question, yes, they are constants.

For more information, you can see these source files: https://github.com/awslabs/aws-data-lake-solution/blob/master/source/cli/core/credentials.js https://github.com/awslabs/aws-data-lake-solution/blob/master/source/cli/core/apiproxy.js

skirk-mpr commented 4 years ago

@beomseoklee Thank you for the prompt reply and the additional information! This is really helpful.

One quick follow up, regarding 4. can you confirm, I see the date in the format YYYYMMDD getting hashed in the provided code samples -- line 28 of credentials.js.

Thanks again!

beomseoklee commented 4 years ago

@skirk-mpr https://github.com/awslabs/aws-data-lake-solution/blob/master/source/cli/core/credentials.js#L28 uses moment, which means it will create a moment date with YYYYMMDD format.

Likely, if you search dateStamp in the documentation, you would see dateStamp is not used at all instead moment is going to be used.

skirk-mpr commented 4 years ago

Thank you @beomseoklee! Looks like I'm all set signing request -- I appreciate your help and guidance. Have a great day!