Firebase Admin SDK: getDownloadUrl - Permission denied. No READ permission

bytewiz commented 8 months ago

Describe your environment

Operating System version: OSX (and in the cloud environment)
Firebase SDK version: ^11.11.0 <= firebase-admin
Firebase Product: firebase-admin / google-storage / firebase-admin/storage
Node.js version: 16.16.0
NPM version: 8.11.0

Describe the problem:

I have now tried for a very long time to follow these docs in order to get getDownloadURL to work. https://firebase.google.com/docs/storage/admin/start#use_a_default_bucket https://firebase.google.com/docs/storage/admin/start#shareable_urls

Regardless of how I initialize my app, when trying to use getDownloadURL I get Error: Permission denied. No READ permission.

Here is how different ways I tried initializing:

initializeApp({
    credential: applicationDefault(),
    storageBucket: "my-bucket.appspot.com",
});

initializeApp();

initializeApp({
    credential: cert(serviceAcount), // loaded from .json file (directly downloaded from firebase console)
    storageBucket: "my-bucket.appspot.com",
});

initializeApp({
    credential: cert({
      projectId: "my-project-id",
      privateKey: "my-private-key",
      clientEmail: "my-client-email"
    }), // grabbed from .json file (directly downloaded from firebase console)
    storageBucket: "my-bucket.appspot.com",
});

Furthermore, I have tried adding IAM roles to the service account:

What I am trying to accomplish is simply what is done in the before-mentioned docs:

    // Triggered from storage.object().onFinalize(generateThumbnail);
    const bucket = getStorage().bucket(object.bucket);
    ...
    // Cloud Storage files.
    const file = bucket.file(filePath);
    const url = await getDownloadURL(file);
    console.log({ url });

What is going wrong here, as the docs states clearly I firebase admin sdk should have access by default?

Stacktrace: (from emulator)

⚠  functions: Error: Permission denied. No READ permission.
    at new ApiError (/../functions/node_modules/firebase-admin/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:80:15)
    at Util.parseHttpRespBody (/../functions/node_modules/firebase-admin/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:215:38)
    at Util.handleResp (/../functions/node_modules/firebase-admin/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:156:117)
    at /../functions/node_modules/firebase-admin/node_modules/@google-cloud/storage/build/src/nodejs-common/util.js:538:22
    at onResponse (/../functions/node_modules/firebase-admin/node_modules/retry-request/index.js:240:7)
    at /../functions/node_modules/firebase-admin/node_modules/teeny-request/build/src/index.js:217:17
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

My service account file:

{
  "type":"..",
  "project_id":"..",
  "private_key_id":"..",
  "private_key":"..",
  "client_email":"..",
  "client_id":"..",
  "auth_uri":"..",
  "token_uri":"..",
  "auth_provider_x509_cert_url":"..",
  "client_x509_cert_url":"..",
  "universe_domain":"..",
}

google-oss-bot commented 8 months ago

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.

tonyjhuang commented 8 months ago

This is the emulator code that's returning this particular error message: https://github.com/firebase/firebase-tools/blob/b7eea76c22816a0caf4e45e6bd0f072c066c5d44/src/emulator/storage/apis/firebase.ts#L111

Unfortunately, our admin credential vetting in the emulator is in a pretty poor state. We don't have full support of OAuth access tokens, which is what the admin SDK is sending along with its requests.

In short the validation flow for getDownloadUrl in the emulator is this:

Does the Authorization header value equal the string literal, "owner"?
If not, validate security rules.

Since there's no way to get the admin SDK to pass along the header "Authorization: Bearer owner", this will likely remain broken until we have a full OAuth validation, which we haven't needed until the introduction of the admin getDownloadURL method.

See this comment at https://github.com/firebase/firebase-tools/blob/master/src/emulator/storage/rules/utils.ts#L81.

This is a true bug and we will get around to fixing this eventually but it's hard to say when the team will have the time to tackle this.

bytewiz commented 8 months ago

This is the emulator code that's returning this particular error message: https://github.com/firebase/firebase-tools/blob/b7eea76c22816a0caf4e45e6bd0f072c066c5d44/src/emulator/storage/apis/firebase.ts#L111

Unfortunately, our admin credential vetting in the emulator is in a pretty poor state. We don't have full support of OAuth access tokens, which is what the admin SDK is sending along with its requests.

In short the validation flow for getDownloadUrl in the emulator is this:

Does the Authorization header value equal the string literal, "owner"?

If not, validate security rules.

Since there's no way to get the admin SDK to pass along the header "Authorization: Bearer owner", this will likely remain broken until we have a full OAuth validation, which we haven't needed until the introduction of the admin getDownloadURL method.

See this comment at https://github.com/firebase/firebase-tools/blob/master/src/emulator/storage/rules/utils.ts#L81.

This is a true bug and we will get around to fixing this eventually but it's hard to say when the team will have the time to tackle this.

@tonyjhuang thanks for taking the time to review it!

But can I rely on that it only persists for the emulator and not in the "real" / production environment when deploying the function? Would the simplest initializeApp(); work or does it require the credentials as of one of the other examples?

Appreciate the support on this!

Or is there any other workaround to get the download url using the firebase-admin sdk in case the above is not working?

bytewiz commented 8 months ago

So does it actually work in production?? @maneesht @tonyjhuang

bytewiz commented 7 months ago

No one here? 😅

weilinzung commented 6 months ago

Same issue with the Firebase emulator, even with custom rules. Only working after deploying to Firebase

    "storage": {
      "port": 9199,
      "rules": "storage-emulator.rules"
    },

rules_version = '2';
service firebase.storage {
  match /b/{bucket}/o {
    match /{allPaths=**} {
      allow read, write: if true;
    }
  }
}

christianbauer1 commented 6 months ago

I have the same issue here and am bypassing the getDownloadUrl in the emulator for now. @tonyjhuang Please notify us if this is fixed 👍

Shakahs commented 6 months ago

Here's my workaround:
Obtain a download token from the emulator's REST API and manually construct an emulator-compatible download URL.

import fetch from 'cross-fetch';
import { FullMetadata } from '@firebase/storage-types';

/**
 * Asynchronously generates and returns the download URL for a file in a specified Firebase Storage emulator bucket.
 * The generated URL can be used to download the file.
 *
 * @param {string} bucket - The name of the Firebase Storage emulator bucket.
 * @param {string} filePath - The path to the file inside the bucket.
 * @returns {Promise<string>} - A promise that resolves to the download URL as a string.
 */
export const getEmulatorDownloadURL = async (bucket: string, filePath: string) => {
    // fetch a new download token
    const tokenGenerationFetch = await fetch(
        `http://${process.env.FIREBASE_STORAGE_EMULATOR_HOST}/v0/b/${bucket}/o/${encodeURIComponent(
            filePath,
        )}?create_token=true`,
        {
            method: 'POST',
            headers: {
                Authorization: 'Bearer owner',
            },
        },
    );
    const tokenGenerationResponse: FullMetadata & { downloadTokens: string } = await tokenGenerationFetch.json();
    const downloadToken = tokenGenerationResponse.downloadTokens.split(',')[0];

    // manually construct the emulator download url
    return `http://${process.env.FIREBASE_STORAGE_EMULATOR_HOST}/v0/b/${bucket}/o/${encodeURIComponent(
        filePath,
    )}?alt=media&token=${downloadToken}`;
};

actuallymentor commented 4 months ago

Is there any debug info we can provide to help make a fix? This is very unfortunate DX.

akselipalmer commented 4 months ago

You can use this function to conditionally run getDownloadUrl() or getEmulatorDownloadURL() based on whether you are running using the firebase emulators or in production.


exports.getFileDownloadUrl = async (filePath) => {
  // Use 'process.env.FUNCTIONS_EMULATOR === "true"' to check your environment.
  // Make sure that "true" is surrounded by quotes because it is a string, not a boolean.
  if (process.env.FUNCTIONS_EMULATOR === "true") {
    // Running using emulators.
    // You can find the bucket in the storage emulator suite.
    // Your bucket name should look something like this: <gs://your-app-name.appspot.com/>.
    return await getEmulatorDownloadURL(bucket, filePath);
  } else {
    // Running in production.
    const fileRef = getStorage().bucket().file(filePath);
    const fileUri = await getDownloadURL(fileRef);
    return fileUri;
  }
};

Here's my workaround: Obtain a download token from the emulator's REST API and manually construct an emulator-compatible download URL.

import fetch from 'cross-fetch';
import { FullMetadata } from '@firebase/storage-types';

/**
 * Asynchronously generates and returns the download URL for a file in a specified Firebase Storage emulator bucket.
 * The generated URL can be used to download the file.
 *
 * @param {string} bucket - The name of the Firebase Storage emulator bucket.
 * @param {string} filePath - The path to the file inside the bucket.
 * @returns {Promise<string>} - A promise that resolves to the download URL as a string.
 */
export const getEmulatorDownloadURL = async (bucket: string, filePath: string) => {
    // fetch a new download token
    const tokenGenerationFetch = await fetch(
        `http://${process.env.FIREBASE_STORAGE_EMULATOR_HOST}/v0/b/${bucket}/o/${encodeURIComponent(
            filePath,
        )}?create_token=true`,
        {
            method: 'POST',
            headers: {
                Authorization: 'Bearer owner',
            },
        },
    );
    const tokenGenerationResponse: FullMetadata & { downloadTokens: string } = await tokenGenerationFetch.json();
    const downloadToken = tokenGenerationResponse.downloadTokens.split(',')[0];

    // manually construct the emulator download url
    return `http://${process.env.FIREBASE_STORAGE_EMULATOR_HOST}/v0/b/${bucket}/o/${encodeURIComponent(
        filePath,
    )}?alt=media&token=${downloadToken}`;
};

kdawgwilk commented 2 months ago

Is there a way to set a custom storage.rules just for the storage emulator? e.g.

{
  ...
  "emulators": {
    "storage": {
      "port": 9199,
      "rules": "storage.rules.emulator"
    },
  }
}

This could be an easy workaround to use open rules for local and then use the regular rules for deployment as a workaround for now.

anonimitoraf commented 2 months ago

@kdawgwilk I haven't tried myself, but you should be able to use a different Firebase config file (ala firebase --config firebase.emulator.json) which references a different storage rules file

jcruzv-prog commented 2 months ago

This solution actually works, is the only way I found out there thanks!

anonimitoraf commented 2 months ago

Btw (for my use case), I found that the download URL was accessible via .publicUrl()

    const uploadRef = storage.bucket().file('assets/' + filename)
    await uploadRef.save(buffer, {
      metadata: { cacheControl: 'public,max-age=86400' },
      public: true,
    })
    return uploadRef.publicUrl()

or via .metadata.mediaLink

    const uploadRef = storage.bucket().file('assets/' + filename)
    await uploadRef.save(file.buffer, {
      metadata: { cacheControl: 'public,max-age=86400' },
      public: true,
    })
    const [metadata] = await uploadRef.getMetadata()
    return metadata.mediaLink

firebase / firebase-admin-node