mapbox / node-pre-gyp

Node.js tool for easy binary deployment of C++ addons
BSD 3-Clause "New" or "Revised" License
1.11k stars 263 forks source link

Can't use Alternative S3 URLs with Staging/Production Option #654

Open ronilan opened 2 years ago

ronilan commented 2 years ago

Overview

It is not possible to use alternative s3 urls (path style) and s3-compatible backends (e.g. min.io) in conjunction with the production_host and staging_host option.

Status

Problem

bucket region and s3ForcePathStyle are set as properties of binary when in fact they are (together with the endpoint) properties of the host target. Since there can be multiple host targets (staging/production) but there can only be a single explicitly set bucket name, the alternate host option doesn't work in conjunction with the production_host and staging_host option.

Proposed Solution

Moving forward host will be defined as an object with optional properties.

Example 1: (S3 Virtual Host URL)

{
  "binary": {
    "host": {
      "endpoint": "https://npg-mock-bucket.s3.us-east-1.amazonaws.com",
    }
  }
}

Example 2: (Path Style alternative host)

{
  "binary": {
    "host": {
      "endpoint": "https://play.min.io",
      "bucket": "node-pre-gyp-production",
      "region": "us-east-1",
      "s3ForcePathStyle": true
    }
  }
}

Example 3: (Utilizing Staging/Production)

{
  "binary": {
    "host": {
      "endpoint": "https://my-production-bucket.s3.us-east-1.amazonaws.com",
    },
    "staging_host": {
      "endpoint": "https://play.min.io",
      "bucket": "node-pre-gyp-staging",
      "region": "us-east-1",
      "s3ForcePathStyle": true
    }
  }
}
{
  "binary": {
    "host": {
      "endpoint": "https://play.min.io",
      "bucket": "node-pre-gyp-production",
      "region": "us-east-1",
      "s3ForcePathStyle": true
    },
    "staging_host": {
      "endpoint": "https://play.min.io",
      "bucket": "node-pre-gyp-staging",
      "region": "us-east-1",
      "s3ForcePathStyle": true
    }
  }
}

Note: the proposed object keys are the same as the ones passed to the AWS sdk.

Backwards compatibility

Existing string definitions and existing keys will be supported.

Example 1: (S3 Virtual Host URL)

{
  "binary": {
    "host": "https://npg-mock-bucket.s3.us-east-1.amazonaws.com",
  }
}

Example 2: (Alternative S3 URL)

{
  "binary": {
    "host": "https://play.min.io",
    "bucket": "node-pre-gyp-staging",
    "region": "us-east-1",
    "s3ForcePathStyle": true
  }
}

Example 3: (Utilizing Staging/Production for S3 Virtual Host URL)

{
  "binary": {
    "staging_host": "https://npg-mock-bucket-staging.s3.us-east-1.amazonaws.com",
    "production_host": "https://npg-mock-bucket-staging.s3.us-east-1.amazonaws.com"
  }
}