feat(storage): add customEndpoint to internal apis in advanced options

[ashwinkumar6]

Description of changes

• Add customEndpoint advanced option to all Internal APIs (7 data plane APIs + 2 control plane APIs).
  • Data plane customEndpoint: https://${customEndpoint}
  • Control plane customEndpoint https://${accountId}.${customEndpoint}
• For list of all possible customEndpoints see s3 service endpoints

Description of how you validated changes

Tested the following code snippets below (internals/). Control plane APIs: listCallerAccessGrants

await listCallerAccessGrants({
  accountId: ACCOUNT_ID,
  region: REGION,
  credentialsProvider: async () => ({ credentials: baseCreds }),
  customEndpoint: `s3-control.dualstack.us-east-2.amazonaws.com`,
maxResults=1000&allowedByApplication=true
});

// GET 200
// https://<accountId>.s3-control.dualstack.us-east-2.amazonaws.com/v20180820/accessgrantsinstance/caller/grants?maxResults=1000&allowedByApplication=true

getDataAccess

await getDataAccess({
  accountId: ACCOUNT_ID,
  region: REGION,
  credentialsProvider: async () => ({ credentials: baseCreds }),
  permission: input.permission,
  scope: input.scope,
  customEndpoint: `s3-control.dualstack.us-east-2.amazonaws.com`,
});

// GET 200
// https://<accountId>.s3-control.dualstack.us-east-2.amazonaws.com/v20180820/accessgrantsinstance/dataaccess?durationSeconds=900&permission=READWRITE&target=....

Data plane APIs: List

await list({
  path: "public/",
  options: {
    customEndpoint: `s3-accelerate.amazonaws.com`,
  },
});

// GET 200
// https://<bucket>.s3-accelerate.amazonaws.com/?list-type=2&max-keys=1000&prefix=public%2F

Copy

await copy({
  source: {
    path: `public/car.jpeg`,
  },
  destination: {
    path: `public/carCopy.jpeg`,
  },
  options: {
    customEndpoint: `s3-accelerate.amazonaws.com`,
  },
});

// PUT 200
// https://<bucket>.s3-accelerate.amazonaws.com/public/carCopy.jpeg

Remove

await remove({
  path: `public/diceCopy.jpeg`,
  options: {
    customEndpoint: `s3-accelerate.amazonaws.com`,
  },
});

// DELETE 204
// https://<bucket>.s3-accelerate.amazonaws.com/public/carCopy.jpeg

Get Properties

await getProperties({
  path: "public/bus.jpeg",
  options: {
    customEndpoint: `s3-accelerate.amazonaws.com`,
  },
});

// HEAD 200
// https://<bucket>.s3-accelerate.amazonaws.com/public/bus.jpeg

Get Url

const result = await getUrl({
  path: `public/bus.jpeg`,
  options: {
    customEndpoint: `s3-accelerate.amazonaws.com`,
  },
});

// GET 200
// https://<bucket>.s3-accelerate.amazonaws.com/public/bus.jpeg?x-amz-content-sha256=....

Download Data

downloadDataTask = downloadData({
  path: "public/bus.jpeg",
  options: {
    customEndpoint: `s3-accelerate.amazonaws.com`,
  },
});
const result = await downloadDataTask.result;

// GET 200
// https://<bucket>.s3-accelerate.amazonaws.com/public/bus.jpeg

Upload Data

const uploadTask = await uploadData({
  path: `public/${file.name}`,
  data: file,
  options: {
    contentType: file.type,
    customEndpoint: `s3-accelerate.amazonaws.com`,
  },
}).result;

// PUT 200
// https://<bucket>.s3-accelerate.amazonaws.com/public/car.jpeg

Data plane APIs:

Checklist

• [x] PR description included
• [x] yarn test passes
• [x] Unit Tests are changed or added
• [ ] Relevant documentation is changed or added (and PR referenced)

Checklist for repo maintainers

• [ ] Verify E2E tests for existing workflows are working as expected or add E2E tests for newly added workflows
• [ ] New source file paths included in this PR have been added to CODEOWNERS, if appropriate

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.