spe-uob / 2020-HealthcareLake

A reasonably secure data lake for healthcare analytics
MIT License
9 stars 5 forks source link

API not working #145

Closed vladbucur2000 closed 3 years ago

vladbucur2000 commented 3 years ago

Describe the bug POST requests on api_url does not work when running multiple datalakes (healthcarelake-demo) (did not test with running a single one so i am not sure where is the problem)

To Reproduce

POST requests within POSTMAN https://ka8fg5y9x1.execute-api.eu-west-2.amazonaws.com/demo : 403 Forbidden https://ka8fg5y9x1.execute-api.eu-west-2.amazonaws.com/demo/Patient : 502 Bad Gateway I used a Bearer token authentication and had the token tested in the API Gateway (aws console).

Expected behavior Code: 201

vladbucur2000 commented 3 years ago

Tried running from the main repo: $ infra % terraform apply var.project_name Enter a value: vlad │Error: Missing required argument │ │ The argument "region" is required, but was not set.

joekendal commented 3 years ago

Tried running from the main repo:

$ infra % terraform apply

var.project_name

Enter a value: vlad

│Error: Missing required argument

│ The argument "region" is required, but was not set.

Apply from the root not ./infra/

That error is related to the absence of the aws provider in infra/ module. The provider is present in ./main.tf so apply can be run from . but not ./infra

joekendal commented 3 years ago

Describe the bug

POST requests on api_url does not work when running multiple datalakes (healthcarelake-demo) (did not test with running a single one so i am not sure where is the problem)

To Reproduce

POST requests within POSTMAN

https://ka8fg5y9x1.execute-api.eu-west-2.amazonaws.com/demo : 403 Forbidden

https://ka8fg5y9x1.execute-api.eu-west-2.amazonaws.com/demo/Patient : 502 Bad Gateway

I used a Bearer token authentication and had the token tested in the API Gateway (aws console).

Expected behavior

Code: 201

  1. Can you confirm your Authorization header doesn't include the prefix "Bearer " in the value

  2. As for the 502, not sure the cause but it's possible that it may take a couple of minutes to become available after deploying.

vladbucur2000 commented 3 years ago
  1. It includes the prefix "Bearer".
  2. I tested it with postman 30 minutes after it was deployed.
joekendal commented 3 years ago

Remove the prefix

georgeedward2000 commented 3 years ago

I tested the API and encountered the following errors:

  1. With "Bearer" prefix and "Patient" as resource: Screenshot 2021-05-11 at 13 25 09
  2. With "Bearer" prefix and wrongly spelled resource - "patient": Screenshot 2021-05-11 at 13 29 25
  3. Without "Bearer" prefix: Screenshot 2021-05-11 at 13 25 59
  4. With "Token" prefix: Screenshot 2021-05-11 at 13 26 28
georgeedward2000 commented 3 years ago

When using Oauth 2.0 in postman, with both "Bearer" prefix and none prefix it is giving "502 Bad Gateway"

georgeedward2000 commented 3 years ago

The token used is tested in API Gateway, Cognito_Authorizer to be sure it is working

joekendal commented 3 years ago

https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-cognito-401-unauthorized/

Try id token and not access token

joekendal commented 3 years ago

Considering our tests are working https://github.com/spe-uob/HealthcareLakeAPI/blob/main/tests/e2e_test.go

I'm going to close this issue for now in the understanding that this is a user error.

Feel free to reopen if you find it is not a user error

vladbucur2000 commented 3 years ago

Only id token was working in the Cognito_authorizer.

However we tried with all of them.

vladbucur2000 commented 3 years ago

I suggest you look at the screenshots from above to see if you see any mistakes in testing the requests to the API.

vladbucur2000 commented 3 years ago

An alternative would be providing us an API (supposing that i am not deploying the infrastructure correctly) just for a small test.

joekendal commented 3 years ago

I have tested it just now and it works. Full steps:

  1. terraform apply
  2. aws cognito-idp initiate-auth --auth-flow USER_PASSWORD_AUTH --auth-parameters USERNAME=testuser,PASSWORD=thepasswordthatwasgenerated, --client-id theclientidthatwasgenerated
  3. Header.Authorization => IdToken
  4. POST url/dev/Patient
vladbucur2000 commented 3 years ago

I have tested it just now and it works. Full steps:

  1. terraform apply
  2. aws cognito-idp initiate-auth --auth-flow USER_PASSWORD_AUTH --auth-parameters USERNAME=testuser,PASSWORD=thepasswordthatwasgenerated, --client-id theclientidthatwasgenerated
  3. Header.Authorization => IdToken
  4. POST url/dev/Patient

That is what I've did.

joekendal commented 3 years ago

I believe the issue could be using credentials that we gave you in the past. The deployment script generates a random password

joekendal commented 3 years ago

Screenshot 2021-05-11 at 12 36 42

joekendal commented 3 years ago

Screenshot 2021-05-11 at 12 37 22

vladbucur2000 commented 3 years ago

I believe the issue could be using credentials that we gave you in the past. The deployment script generates a random password

Located in the terraform.tfstate , right? There is no issue in obtaining the token.

joekendal commented 3 years ago

And you have only set the header Authorization with no bearer prefix and no postman authorization flow This was working fine

georgeedward2000 commented 3 years ago

Yep "No Auth" was tested as well. Everything is on point. We tried both cookie cutter and single datalake deployments. None worked.

joekendal commented 3 years ago

Are you testing one data lake or using the cookie cutter thing? We identified a naming conflict that would occur when applying multiple lakes to one account in https://github.com/spe-uob/HealthcareLakeAPI/blob/9e0461c4f4a3eec97cd55ec64b73a07d1d2ab026/modules/api_gateway/main.tf#L3

Multiple lakes are not officially supported by this module or the API module at this stage, anything you are working on there is experimental only. There is no official release

vladbucur2000 commented 3 years ago

In conclusion HealthcareLakeDemo is not working?

There are too many repos and I find it very confusing. Which one is the one that we should deploy and test on it?

joekendal commented 3 years ago

Hi, Demo is not an official project supported by our team. This repository is the parent module for our Terraform project. You can receive support for HealthcareLake or its dependencies (HealthcareLakeAPI and HealthcareLakeETL)

joekendal commented 3 years ago

You can deploy this one. Please refer to the README to answer your questions about dependencies.

joekendal commented 3 years ago

You need to have the api key as a header with the name "x-api-key" and the id_token needs to be put into the Authoriztion menu having selected "Bearer Token"

There is no need for api key. This has been made redundant. If you select No Auth and set Authorization header to IdToken it would work as shown in my screenshots. But Demo project is unsupported/experimental

joekendal commented 3 years ago

Since this issue has been resolved, the thread will be locked to prevent further confusion. Please feel free to reach out if it is still unclear