[RFE] Be able to set endpoint for local DB usage

[QAnders]

I've been testing a bit using a local Docker data-api which works great using the AWS SDK. It would be awesome to be able to test locally using the data-api-client against the local DB!

docker run --rm -it --name my-data-api -p 8080:80 -e ENGINE=PostgreSQLJDBC -e POSTGRES_HOST=host.docker.internal -e POSTGRES_PORT=5432 -e POSTGRES_USER=anders -e POSTGRES_PASSWORD=password -e RESOURCE_ARN=arn:aws:rds:us-east-1:123456789012:cluster:dummy -e SECRET_ARN=arn:aws:secretsmanager:us-east-1:123456789012:secret:dummy koxudaxi/local-data-api

This spins up a local Docker that in turn points to my local PosgreSQL DB.

The following code works fine and executes the data-api against the local DB:

const rdsdataservice = new AWS.RDSDataService({
  apiVersion: '2018-08-01',
  endpoint: 'http://127.0.0.1:8080',
  region: 'us-east-1'
});
const r= rdsdataservice.executeStatement({
  sql: 'select * from "user"',
  resourceArn: 'arn:aws:rds:us-east-1:123456789012:cluster:dummy',
  secretArn: 'arn:aws:secretsmanager:us-east-1:123456789012:secret:dummy',
  database: 'test_db'
}).promise();
r.then( ret => {
  console.log(ret.records)
});

I can't however figure out how this would be invoked using data-api-client... I've tried to set the parameters exactly as in the sample in data-api-client but something is not 100% as I get an error stating: HttpEndPoint is not enabled for arn:aws:secretsmanager:us-east-1:123456789012:secret:dummy

That error is not present when using the AWS.RDSDataService natively in the sample so there must be some "behind the scenes" settings/parameter that differs that I simply can't figure out...

Just a request for an enhancement! :)

Thanks for your work!

[jeremydaly]

Hi @QAnders. I didn't even realize you could do this! It's likely that the library is just ignoring your endpoint configuration as it would be filtering the inputs. It might just be a matter of adding an additional configuration field to specify your localhost. If it works with the AWS SDK, then there's no reason why it can't work here. Let me take a look.

[healthpraxone]

I had faced the same problem and I actually tried to create a fork and added a separate param to the endpoint https://github.com/healthprax/mysql-data-api and it works fine with this minor change

[QAnders]

Turns out my issues was with other AWS settings, so managed to get it going with a very simple addition.

My "issue" why I couldn't get it working was because I am using Serverless and I have other "online" AWS services, e.g. SSM (Parameter Store) and S3 that is setup using environment variables and then some "behind-the-scenes" AWS SDK thingies was not happy as I had set the Docker "dummy" to other parameters.

I had to alter the values of the local-data-api Docker instance to actually match my "real" set AWS account ID and region and then restart the Docker container and after that it runs fine!

29xxxxxxxxx is my real AWS account ID:

docker run --rm -it --name my-data-api -p 8080:80 -e ENGINE=PostgreSQLJDBC -e POSTGRES_HOST=host.docker.internal -e POSTGRES_PORT=5432 -e POSTGRES_USER=anders -e POSTGRES_PASSWORD=password -e RESOURCE_ARN=arn:aws:rds:eu-west-1:29xxxxxxxxx:cluster:dummy -e SECRET_ARN=arn:aws:secretsmanager:eu-west-1:29xxxxxxxxx:secret:dummy koxudaxi/local-data-api

Then just add endpoint to the initial options in index.js , like:

  // Update the AWS endpoint to enable local DB/Docker container
  if (typeof params.endpoint === 'string') {
    options.endpoint = params.endpoint
  }

And then initiate data-api-client like:

const rdsds = require('data-api-client')({
  secretArn: 'arn:aws:secretsmanager:eu-west-1:29xxxxxxxxx:secret:dummy', //process.env.RDS_AURORA_DATA_API_SECRET_ARN,
  resourceArn: 'arn:aws:rds:eu-west-1:29xxxxxxxxx:cluster:dummy', //process.env.RDS_AURORA_DATA_API_ARN,
  database: 'qip_db', //process.env.RDS_AURORA_DATA_API_QIP_DB,
  region: 'eu-west-1',
  endpoint: 'http://127.0.0.1:8080'
});

[QAnders]

Would it be possible to get the endpointparameter into the options, you think?

 // Update the AWS endpoint to enable local DB/Docker container
  if (typeof params.endpoint === 'string') {
    options.endpoint = params.endpoint
  }

[jonocairns]

@QAnders is it possible to just use the options override to do this?

e.g.

const db = DataApiClient({
  resourceArn: 'arn:aws:rds:us-west-2:123456789012:cluster:dummy',
  secretArn: 'arn:aws:secretsmanager:us-west-2:123456789012:secret:dummy',
  database: 'dbName',
  options: {
    endpoint: 'http://docker.for.win.localhost:8090',
  },
});

[QAnders]

Indeed! Thanks @jonocairns that works like a charm!

const db = require('data-api-client')({
  secretArn: process.env.RDS_AURORA_DATA_API_SECRET_ARN,
  resourceArn: process.env.RDS_AURORA_DATA_API_ARN,
  database: process.env.RDS_AURORA_DATA_API_QIP_DB,
  region: 'eu-west-1',
  options: {
    endpoint: process.env.IS_OFFLINE && ['dev', 'local'].includes(process.env.LAMBDA_STAGE)
      ? 'http://127.0.0.1:8080'
      : null
  }
});