Use a single bucket for Huly

NiklasRosenstein commented 4 months ago

Description of the issue

As far as I can discern, it appears that Huly tries to create a new S3 bucket for each Workspace. I'm encountering this because I have an existing MinIO instance, there doesn't seem to be an option to specify any single bucket and creating a workspace fails with this error:

update { createProgress: 20 }
create minio bucket workspace="w-niklas-test-665ccad3-e545e9bc94-ba007d" workspaceId={"name":"w-niklas-test-665ccad3-e545e9bc94-ba007d","productId":""}
failed to create workspace err={"name":"AIe","code":"AccessDenied","message":"Access Denied.","bucketname":"w-niklas-test-665ccad3-e545e9bc94-ba007d","resource":"/w-niklas-test-665ccad3-e545e9bc94-ba007d","requestid":"17D547561B44D5F5","hostid":"dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8","amzRequestid":"17D547561B44D5F5","amzId2":"dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8"} workspaceName="test" email="niklas@rosenstein.email"
Unhandled Rejection at: Promise {
  <rejected> AIe: Access Denied.
      at zVt (/usr/src/app/bundle.js:411:4230)
      at Object.D$n [as parseResponseError] (/usr/src/app/bundle.js:411:4917)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async nt.makeRequestStreamAsync (/usr/src/app/bundle.js:417:2307)
      at async nt.getBucketRegionAsync (/usr/src/app/bundle.js:417:2785)
      at async nt.makeRequestStreamAsync (/usr/src/app/bundle.js:417:1709)
      at async nt.makeRequestAsyncOmit (/usr/src/app/bundle.js:417:789) {
    code: 'AccessDenied',
    bucketname: 'w-niklas-test-665ccad3-e545e9bc94-ba007d',
    resource: '/w-niklas-test-665ccad3-e545e9bc94-ba007d',
    requestid: '17D547561B44D5F5',
    hostid: 'dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8',
    amzRequestid: '17D547561B44D5F5',
    amzId2: 'dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8',
    amzBucketRegion: undefined
  }
} reason: AIe: Access Denied.
    at zVt (/usr/src/app/bundle.js:411:4230)
    at Object.D$n [as parseResponseError] (/usr/src/app/bundle.js:411:4917)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async nt.makeRequestStreamAsync (/usr/src/app/bundle.js:417:2307)
    at async nt.getBucketRegionAsync (/usr/src/app/bundle.js:417:2785)
    at async nt.makeRequestStreamAsync (/usr/src/app/bundle.js:417:1709)
    at async nt.makeRequestAsyncOmit (/usr/src/app/bundle.js:417:789) {
  code: 'AccessDenied',
  bucketname: 'w-niklas-test-665ccad3-e545e9bc94-ba007d',
  resource: '/w-niklas-test-665ccad3-e545e9bc94-ba007d',
  requestid: '17D547561B44D5F5',
  hostid: 'dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8',
  amzRequestid: '17D547561B44D5F5',
  amzId2: 'dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8',
  amzBucketRegion: undefined
}

I was hoping to just give it a huly-data bucket. Alternatively, is there a way to configure the prefix that is used when Huly creates a new bucket? At the moment that appears to be w, though I would like to choose something more distinctive among my other buckets.

Your environment

Version of huly: v0.6.230

Steps to reproduce

Give Huly access credentials to MinIO that can't create a bucket.

haiodo commented 2 months ago

for s3 is possible to specify rootBucket variable to pass all workspaces into one bucket.

NiklasRosenstein commented 2 weeks ago

Hi @haiodo , thanks I'm looking into this again right now. How do I set this configuration value? There doesn't seem to be an environment variable that is used for this option.

DyllasDek commented 1 day ago

Any movement on that?

hcengineering / platform