Feature Request: support CORS config in Storage emulator

andriibulbuk commented 1 year ago

[REQUIRED] Environment info

11.23.0

firebase-tools:

macOS

Platform:

[REQUIRED] Test case

I have two separate repositories with my node.js API and my Firebase repository. My API runnings outside of Firebase environment and I use the "firebase-admin" package. So first of all I start my firebase emulator services (ui, functions, firestore, storage). And I can access my "olympus-grip-dev.appspot.com" bucket from emulator UI, but when I try to connect to firebase storage from my api I get APIError: NotImplemented on node v16.17.0 and FetchError on node v18.14.2. I can access firestore emulator and cloud functions emulator works well with my api.

[REQUIRED] Steps to reproduce

Run firebase emulators (ui, functions, firestore, storage)
Run express application:

admin.initializeApp({ projectId: GCLOUD_PROJECT });

envs: APP_PORT=3000 INSTANCE_APP_PORT=3000 GCLOUD_PROJECT=olympus-grip-dev GOOGLE_APPLICATION_CREDENTIALS=/Users/andriibulbuk/Projects/OG-admin-api/src/common/configs/google-application-credentials.json FIRESTORE_EMULATOR_HOST=localhost:8080 FIREBASE_STORAGE_EMULATOR_HOST=localhost:9199

[REQUIRED] Expected behavior

My API successfully connects to the firebase storage emulator

[REQUIRED] Actual behavior

Get such errors when try to connect from api to firebase storage:

v16.17.0: 1] ApiError: Not Implemented [1] at new ApiError (/Users/andriibulbuk/Projects/OG-admin-api/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:79:15) [1] at Util.parseHttpRespMessage (/Users/andriibulbuk/Projects/OG-admin-api/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:181:41) [1] at Util.handleResp (/Users/andriibulbuk/Projects/OG-admin-api/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:155:76) [1] at /Users/andriibulbuk/Projects/OG-admin-api/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:537:22 [1] at onResponse (/Users/andriibulbuk/Projects/OG-admin-api/node_modules/retry-request/index.js:240:7) [1] at /Users/andriibulbuk/Projects/OG-admin-api/node_modules/teeny-request/build/src/index.js:226:13 [1] at processTicksAndRejections (node:internal/process/task_queues:96:5) { [1] code: 501...

v18.14.2 [1] FetchError: request to http://localhost:9199/b/olympus-grip-dev.appspot.com? failed, reason: connect ECONNREFUSED ::1:9199 [1] at ClientRequest. (/Users/andriibulbuk/Projects/OG-admin-api/node_modules/node-fetch/lib/index.js:1505:11) [1] at ClientRequest.emit (node:events:513:28) [1] at Socket.socketErrorListener (node:_http_client:502:9) [1] at Socket.emit (node:events:513:28) [1] at emitErrorNT (node:internal/streams/destroy:151:8) [1] at emitErrorCloseNT (node:internal/streams/destroy:116:3) [1] at process.processTicksAndRejections (node:internal/process/task_queues:82:21) { [1] type: 'system', [1] errno: 'ECONNREFUSED', [1] code: 'ECONNREFUSED' [1] }

[2023-05-23T11:50:04.325Z] Ignoring unsupported arg: port {"metadata":{"emulator":{"name":"storage"},"message":"Ignoring unsupported arg: port"}} [2023-05-23T11:50:04.748Z] Temp file directory for storage emulator: /var/folders/b6/5ymqsg8934q7h6ttvcwgb8780000gn/T/firebase/storage/blobs {"metadata":{"emulator":{"name":"storage"},"message":"Temp file directory for storage emulator: /var/folders/b6/5ymqsg8934q7h6ttvcwgb8780000gn/T/firebase/storage/blobs"}}

christhompsongoogle commented 1 year ago

This line makes me think it's a DNS resolution issue:

[1] FetchError: request to http://localhost:9199/b/olympus-grip-dev.appspot.com? failed, reason: connect ECONNREFUSED ::1:9199

Try using 127.0.0.1 instead of localhost and let me know if that resolves this issue on 18

andriibulbuk commented 1 year ago

Changed localhost to IP address 127.0.0.1 in my .env file. Now get this error on both versions of node:

[1] ApiError: Not Implemented [1] at new ApiError (/Users/andriibulbuk/Projects/OG-admin-api/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:79:15) [1] at Util.parseHttpRespMessage (/Users/andriibulbuk/Projects/OG-admin-api/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:181:41) [1] at Util.handleResp (/Users/andriibulbuk/Projects/OG-admin-api/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:155:76) [1] at /Users/andriibulbuk/Projects/OG-admin-api/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:537:22 [1] at onResponse (/Users/andriibulbuk/Projects/OG-admin-api/node_modules/retry-request/index.js:240:7) [1] at /Users/andriibulbuk/Projects/OG-admin-api/node_modules/teeny-request/build/src/index.js:226:13 [1] at process.processTicksAndRejections (node:internal/process/task_queues:95:5) { [1] code: 501,

andriibulbuk commented 1 year ago

My firebase.json file:

{
  "functions": {
    "predeploy": [
      "npm --prefix \"$RESOURCE_DIR\" run build"
    ],
    "source": "./"
  },
  "emulators": {
    "ui": {
      "enabled": true,
      "port": 4000,
      "host": "0.0.0.0"
    },
    "functions": {
      "port": 5001,
      "host": "0.0.0.0"
    },
    "firestore": {
      "port": 8080,
      "host": "0.0.0.0"
    },
    "storage": {
      "port": 9199,
      "host": "0.0.0.0"
    }
  },
  "storage": {
    "rules": "storage.rules"
  }
}

storage.rules:

rules_version = '2';
service firebase.storage {
  match /b/{bucket}/o {
    match /{allPaths=**} {
      allow read, write: if request.auth!=null;
    }
  }
}

Command to start emulators:

firebase emulators:start --import ./src/mockup

aalej commented 1 year ago

Hi @andriibulbuk. Let me know if I may have misunderstood anything. From what I can tell you are running an Express app which uses the "firebase-admin" sdk to access the Firebase Storage Emulator.

I tried to reproduce the behavior you mentioned but I did not encounter any issues when accessing files in the Firebase Storage Emulator using my Express app. I’m not sure if I may be missing something, but would it be okay if you could provide code snippets of how you access the Firebase Storage Emulator or an MCVE that we can run locally?

Also, I noticed you mentioned you were using firebase-tools v11.23.0; are you able to replicate the issue when using the latest version(v12.2.0)?

andriibulbuk commented 1 year ago

It seems that an error happens when I try to configure cors for my bucket:

  await this.storage
        .bucket(this.config.bucket as string)
        .setCorsConfiguration([
          {
            maxAgeSeconds: 3600,
            method: ["GET"],
            origin: ["*"],
            responseHeader: ["Content-Type"]
          }
        ]);

Without this configuration I can connect to the Firebase storage emulator

aalej commented 1 year ago

Based on our docs, it looks like Cloud Storage for Firebase emulator currently does not support bucket-level CORS configuration. This may be the reason why you’re encountering the error when using setCorsConfiguration. I also confirmed this on my end, using setCorsConfiguration raises an error when I try running my app.

The Cloud Storage for Firebase emulator does not support any bucket-level configuration including storage class, bucket-level CORS configuration, labels, or retention policies. Firebase intends to improve this support over time.

That said, I’ll be marking this issue as a feature request. Please note that we would not be able to provide a timeline as to when this would be implemented. I can only suggest checking this ticket for updates.

google-oss-bot commented 1 year ago

Hey @andriibulbuk. We need more information to resolve this issue but there hasn't been an update in 7 weekdays. I'm marking the issue as stale and if there are no new updates in the next 3 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

dr-aiuta commented 1 year ago

same issue here, I think it's related to #4741

what appears to be a solution: https://github.com/firebase/firebase-admin-node/issues/2084#issuecomment-1442663564

aalej commented 1 year ago

Thanks for the inputs @dr-aiuta. Seems like the comment in firebase/firebase-admin-node#2084 (comment) resolves the dns related issue when using Node 18. However, even after replacing localhost with 127.0.0.1, using the code snippet provided in https://github.com/firebase/firebase-tools/issues/5881#issuecomment-1563027826 will raise an error.

If in case you are still encountering an issue connecting to the Cloud Storage for Firebase emulator, and you’re not using setCorsConfiguration. I would recommend filing a new ticket as your issue might be different from the one filed here.

firebase / firebase-tools