Aurora Postgres Severless Data API Support

[juliedavila]

Any interest in supporting it? I'm happy to help.

https://docs.aws.amazon.com/rdsdataservice/latest/APIReference/API_ExecuteStatement.html

[mcollina]

I think a PR would be nice. How would you implement it?

[juliedavila]

High level

• TrailsManager constructor => add check for an aurora settings in config, error if aurora setttings are simultaneously set with host, port, etc; this would obv make it so the pg.Pool isn't invoked as that's not a thing when using the DataAPI
• performDatabaseOperations would be extended to do the same building logic but in a format needed for the REST call.
  • this would likely get a bit bloated so might put that logic in lib/aws.js or something

It goes without saying that this would require some tinkering to check perf and other bits. You can already use Trails against Aurora but it requires having Lambada functions to run in a VPC which is a perf hit. The DataAPI gets you around this restriction and alleviates pool management.

The only caveat is that it would be on the "end" user to configure their DB/network/etc. Though I'd be happy to provide a sample SAM/CloudFormation template that would provide a POC/sandbox for people to experiment with.

[juliedavila]

thinking further, we could make pg import conditional too, since we'll be needing to add another dependency on the AWS SDK.

The AWS SDK is quite large, but there are some options to make it smaller and "only what's needed"

1. AWS-provided SDK builder https://sdk.amazonaws.com/builder/js/ this would mean placing the build in a local node_modules which you might not want.
2. A more hands-off approach @aws-sdk/client-rds-data-node

[juliedavila]

I'll hold off working on this until your review of my general approach.

[mcollina]

How about a slightly different approach that does not involve adding any deps, but rather enable you to create another module that can be used?

Something like a SQLProvider class that offers an execute method that takes the output of SQL? The internal implementation could be kept as it is internally, and you can develop the Aurora integration on a separate package.

[juliedavila]

Oh, that's a good idea, sure. I can go down that path.

On Thu, Mar 05, 2020 at 10:21 AM, Matteo Collina < notifications@github.com > wrote:

How about a slightly different approach that does not involve adding any deps, but rather enable you to create another module that can be used?

Something like a SQLProvider class that offers an execute method that takes the output of SQL ( https://github.com/nearform/sql ) ? The internal implementation could be kept as it is internally, and you can develop the Aurora integration on a separate package.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub ( https://github.com/nearform/trail/issues/18?email_source=notifications&email_token=ABDSWYXUQIAUOFFGCBJLJNDRF67QLA5CNFSM4K55O7SKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEN5VN3Y#issuecomment-595285743 ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/ABDSWYQJ4CEHIW6JPV563XDRF67QLANCNFSM4K55O7SA ).

[juliedavila]

I think we can close the RFE.

For now the best path is to just use Aurora in a VPC context and connect to it as you would any other db.

The reason, the AWS Aurora DataAPI doesn't currently support parameterized queries which means using the @nearform/sql lib is not really doable and the alternative is probably not too secure.

Apologies for not realizing that earlier, I noticed as I was working on implementing the above.

[mcollina]

No worries, thanks!