Let API talk to Tasking Manager RDS

hotosm / galaxy-api

Backend to fetch data from Underpass

https://galaxy-api.hotosm.org/latest/redoc

GNU Affero General Public License v3.0

14 stars 5 forks source link

Let API talk to Tasking Manager RDS #171

Closed ramyaragupathy closed 2 years ago

ramyaragupathy commented 2 years ago

From discussion here https://github.com/hotosm/galaxy-api/issues/153#issuecomment-1054069415

Galaxy API should be able to connect to Tasking Manager RDS. Challenge comes from the resources living in different VPCs.

@eternaltyro is working on resolving the issue

cc @itskshitiz321 @JorgeMartinezG

eternaltyro commented 2 years ago

Components

Security group rules for TM RDS (allowing connections from Galaxy API security group)
Peering connection between the two VPCs (w/ Private DNS hostnames and private addresses enabled)

eternaltyro commented 2 years ago

Both of these are done from the infrastructure side. I am currently setting up the container to connect to the database with the right credentials (See comments in #140)

Testing the connection:

Log files - container logs exported to cloudwatch logs
Make a call to the Galaxy API and verify results

eternaltyro commented 2 years ago

The new API service has been deployed to docker containers on AWS ECS and has been made temporarily accessible on galaxy-api-lb.hotosm.org

To be production worthy, the following needs to happen

[x] The API endpoint needs to be tested to verify that data is being returned for queries targeting data from:
- [x] Tasking Manager Database
- [x] Insights Database
- [x] Underpass Database
[x] Point galaxy-api.hotosm.org to the container load-balancer and remove galaxy-api-lb.hotosm.org
[x] Wait for DNS propagation and remove osm-stats.hotosm.org
[x] Update the website with the new endpoint

I am hoping to get some support from an AWS Solutions Architect regarding the AWS Cloudfront redirection issue that would let us have galaxy.hotosm.org/api/... but until that happens, let's work with galaxy-api.hotosm.org

eternaltyro commented 2 years ago

@d-rita Please help test the endpoints and post results of your test here if possible. Thanks.

d-rita commented 2 years ago

With the galaxy-ui:

✅ I connected my local frontend app (galaxy-ui) to https://galaxy-api.hotosm.org and was able to make a request to the https://galaxy-api.hotosm.org/mapathon/summary endpoint. I got back the data.

With the API docs:

✅ While using the docs themselves, i.e. https://galaxy-api.hotosm.org/docs, I was able to retrieve OSM ids from the osm-users/ids endpoint.
⛔️ However, I am yet to test out the authenticated routes because I cannot seem to get a token yet. When I sign in to OSM with the url I get back from the /login endpoint, I get an error: The requested redirect uri is malformed or doesn't match client redirect URI. I do need some help with this part before I can go further. cc: @eternaltyro

d-rita commented 2 years ago

@eternaltyro, these are the steps I took:

Go to the login endpoint via docs.
Copy the URL response returned by the endpoint, and enter it into a new tab.
Logged in via OSM.
Got an error:

eternaltyro commented 2 years ago

The callback URL domain should match the actual domain from which OAuth requests are made. I fixed this in the current Docker setup.