Put User Workspaces REST API behind AWS API Gateway

[yuanzhou]

Endpoints from Juan:

POST /tokens/

GET /workspaces/
GET /workspaces/<workspace_id>
GET /workspaces/?query_params
POST /workspaces/
PUT /workspaces/<workspace_id>

GET /jobs/
GET /jobs/<job_id>
GET /jobs/?query_params
PUT /jobs/<job_id>

GET /job_types/

Two items need further attention:

1. I'm not sure how AWS API Gateway would handle the endpoints with query string. Things like

   GET /jobs/
   GET /jobs/?query_params

   will need to be defined as the same resource. Maybe I won't need to worry about GET /jobs/?query_params ...

2. And in terms of security/access control, I'm not sure how AWS API Gateway would integrate with the PSC.

[yuanzhou]

Resource from Matt: https://aws.amazon.com/blogs/security/how-to-enhance-amazon-cloudfront-origin-security-with-aws-waf-and-aws-secrets-manager/

[yuanzhou]

• Add WAF ACL and attach to API Gateway to control IPs
• Create lambda function to forward requests to API running on PSC VM
• Create NAT in the current VPC with private subnet and associate with the lambda function
• PSC VM only allow the Elastic IP of the private subnet

[yuanzhou]

More resources:

• https://aws.amazon.com/blogs/compute/using-aws-lambda-iam-condition-keys-for-vpc-settings/
• https://ao.ms/the-provided-execution-role-does-not-have-permissions-to-call-createnetworkinterface-on-ec2/
• https://bobbyhadz.com/blog/aws-lambda-provided-execution-role-does-not-have-permissions

[shirey]

After further review and discussion with Juan we've decided to not put this behind the AWS Gateway.

[yuanzhou]

From Juan:

Domain split between HTTP/WS/“Passthrough” Passthrough here refers to the traffic that has to go through to JupyterLab (and any other interactive software) HTTP/WS would live under AWS API Gateway Passthrough traffic would have to be done outside of the context of the AWS API Gateway Traffic would have to be opened up to the public for that particular “passthrough” domain. Development effort would have to be spent on modifying the passthrough URLs. Regardless of the option that we choose here, we wouldn’t be able to leverage the AWS API Gateway Globus authentication for the “passthrough” functionality anyway, as I don’t have any control over the headers/cookies that JupyterLab sets.

we can proceed with setting up AWS API REST/Websockets for everything except the /passthrough routes. They can have different domain names.

We'll have the following domains:

• ws-workspaces-api.dev.hubmapconsortium.org and workspaces-api.dev.hubmapconsortium.org on DEV
• ws-workspaces.api.hubmapconsortium.org and workspaces.api.hubmapconsortium.org on PROD

[shirey]

This is complete for the RESTful endpoints, will track the work for web sockets here: https://github.com/hubmapconsortium/devops/issues/21

[yuanzhou]

Update: AWS API Gateway doesn't support websocket proxy to another backend websocket. We'll be using nginx on-prem to handle the call on PSC. PSC will get new domain and ssl cert for this.

[yuanzhou]

Working version on DEV: https://github.com/hubmapconsortium/devops/issues/21#issuecomment-1126189744