can't run "firebase serve" when offline

[ultraGentle]

Hello,

Steps to reproduce: try to run "firebase serve" from the command line while offline
Yields: "Error: Authentication Error: Your credentials are no longer valid. Please run firebase login --reauth"

If I go back online, the "serve" commands works as expected; no need to actually re-authenticate.
Effectively, this means I can only use "serve" while online, which is a bummer.

firebase-tools 7.9.0 macOS 10.13.6

[google-oss-bot]

This issue does not seem to follow the issue template. Make sure you provide all the required information.

[ultraGentle]

update: And, if I go online to successfully launch the emulator, going offline crashes it with "Error: An unexpected error has occurred."

This did not happen before 7.9.0
Is there a procedure to roll back to a previous version of firebase-tools?

[samtstern]

@ultraGentle if you want to roll back to a specific version you can declare it in the install command:

npm install -g firebase-tools@7.8.0

Are you trying to serve hosting, functions or both? And do you remember the last version of the CLI that worked offline for you?

[ultraGentle]

Re: if you want to roll back
I tried rolling back to 6.12; same error

Re: trying to serve hosting, functions, or both?
Both. Neither "firebase serve" nor with "--only hosting,functions" works.

Re: last version of the CLI that worked offline
I couldn't tell you by number, but by rough date, I'd say whatever was out in April 2019 worked offline for sure.

Then, less certain, I had thought that early November 2019 worked offline -- I certainly didn't receive the aforementioned error in November -- but it's possible that I coincidentally had an internet connection each time running "firebase serve."

I'm happy to do additional testing / diagnostics, if you point me in the right direction.

[ultraGentle]

Update: I tried firebase emulators:start, and it, too, fails with the same errors when offline. Let me know if you'd like me to open a separate issue for that, or possibly change the title of this one.

[samtstern]

@ultraGentle I looked into this and this is the intended behavior for the hosting emulator (currently) and has been for some time. You can run the Functions, Firestore, and Realtime Database emulators offline.

It's used to power the behavior where __init.js magically calls firebase.initializeApp() with your app's config.

However I do think it would be reasonable to just skip this step when offline and log a warning instead. So I will change this to a feature request.

[samtstern]

This function:

module.exports = function(options) {
  return fetchWebSetup(options).then(function(config) {
    var configJson = JSON.stringify(config, null, 2);
    return {
      js: INIT_TEMPLATE.replace("{/*--CONFIG--*/}", configJson),
      json: configJson,
    };
  });
};

Could be changed in a number of ways to be more offline-friendly:

1. When offline replace the whole firebase.initializeApp() call with just a console.warn() call.
2. Cache the result of fetchWebSetup and use it when offline, so you only have to be online once.

[ultraGentle]

@samtstern I appreciate the response. That makes sense. I like your suggestion number 2 (cache the config), especially.

I'll just note, in case it slipped through the cracks, that the hosting emulator crashes if you go offline, even after it has begun hosting, at which point it has already fetched the config.

In the meanwhile, if there's a way of manually inserting the config into the hosting emulator, that would be useful info. But otherwise, no response requested, and thanks for not only looking into this but coming up with a couple of nice possible solutions!

[samtstern]

@ultraGentle you could not include __init.js and instead just call firebase.initializeApp() yourself but the emulator still wouldn't start up offline because the fetchWebSetup call is in the startup flow.

[filipesilva]

Another consequence of this behaviour is that using the emulator requires login and permissions, as described in https://github.com/firebase/firebase-tools/issues/1861. I'll copy the relevant bits since it's a duplicate:

---

Hi there 👋

I'm trying to setup the emulators on a couple of projects but am running into a snag. It seems like the hosting emulator requires both login and certain permissions on the project, whereas the other emulators don't. I think this is a bug because the behaviour runs contrary to the purpose and use cases of the emulators.

This is a bit of a problem for me as a developer, and member of a larger team, because it means I must have firebase project permissions I wouldn't need otherwise. In fact I shouldn't have these permissions at all.

It is also a problem for CI environments because now the CI machine also needs to both login and have permissions, which implies some mechanism for secret sharing. I'd also need to have access to CI configuration to add the secret and keep it up to date. It's common enough for CI providers to support secret sharing, but it sounds like an unnecessary attack vector.

Lastly this also makes it hard to provide reproducible environments for open source projects. The repro provided (https://github.com/filipesilva/firebase-emulator-test) will work for me but not for anyone else.

The workaround is to use all emulators but the hosting one, and then use something else to serve (like the npm serve package). When using this workaround, I also need to match any firebase functionality like public and rewrites on the replacement server. This is a bit onerous and hurts the fidelity of the testing environment, but works.

[filipesilva]

@samtstern following your comments about offline capability, do you also consider it reasonable to just log a warning when authentication isn't possible, even if online?

[samtstern]

@filipesilva since the blocking code path here is all about convenience, I do think it's reasonable to never consider it an error and always consider failure a warning. We just have to make very clear warning messages that distinguish being offline from a real authentication error or a backend error that could be retried.

[filipesilva]

Awesome, is this something that the team is open to a contribution to address or is it best to wait?

[samtstern]

@filipesilva I hate to discourage a contribution (in general we love them!) but in this case I want to ask a few people on the Hosting team what they recommend. We have to be super careful messing with any behavior of serve because it's such a widely used command with such a long history.

[samtstern]

Ok so I was able to easily implement caching of the Firebase config (https://github.com/firebase/firebase-tools/commit/3ffaacde2567d1e587a2577437ab511bd7c41f5b) but the new problem is the Firebase SDK.

By default Firebase Hosting puts tags like this on your page:

<script defer src="/__/firebase/7.6.0/firebase-auth.js"></script>
<script defer src="/__/firebase/7.6.0/firebase-database.js"></script>
<script defer src="/__/firebase/7.6.0/firebase-messaging.js"></script>
<script defer src="/__/firebase/7.6.0/firebase-storage.js"></script>

However those scripts are not actually bundled, they're served from the CDN at https://www.gstatic.com/firebasejs/ so those will not load when you're offline which makes the app somewhat useless (unless you happen to have a browser cache).

Anyone got a clever idea or workaround for this?

[filipesilva]

So those urls are proxied to the CDN ones E.g. /__/firebase/7.6.0/firebase-auth.js is proxied to https://www.gstatic.com/firebasejs/__/firebase/7.6.0/firebase-auth.js? If you're already proxying them then you can also proxy them to versions in node_modules, if present. Last resort can be serving nothing at all on those urls, or not injecting them at all.

I didn't know this happened so it is somewhat surprising to me. For testing scenarios with the emulators, I think adding them isn't great. It'll skew metrics such as lighthouse and might introduce other unexpected behaviour. I don't know if there's a smart way to go about it, but it does have drawbacks.

[filipesilva]

I couldn't see those tags when using the emulators in https://github.com/filipesilva/firebase-emulator-test though. The served html looks exactly like the one I have in https://github.com/filipesilva/firebase-emulator-test/blob/master/public/index.html.

[ultraGentle]

My use-case for offline hosting is very simple: (1) I often just need to open a browser for manual interaction with a Firebase web app during design and development, and (2) I don't always have a reliable internet connection. So for me, even an inaccurate (re: lighthouse) offline hosting environment would be sufficient for development. (That is, if I understood the comment about browser cache correctly.)

[samtstern]

Thanks all for the comments! I have a proposal out in #1864 and would love your feedback.

@filipesilva we don't always add those tags to your page, but it is included in the default firebase init hosting template. In production those __/firebase/* URLs are served directly from the production hosting server on your own domain which allows them to share the same HTTP2 connection, leading to a slightly more efficient load than loading from a CDN.

@ultraGentle thanks for sharing! Would it be ok if offline support was only added to emulators:start and not serve? I am extremely hesitant to change something fundamental about serve at this point and emulators:start is where we hope to do the future of Firebase local development.

[filipesilva]

Oh awesome, I wasn't expecting a PR so fast!

Is there a snapshot I can use for local testing? Actually, since these are just JS files, I can copy them directly into my node_modules.

[filipesilva]

Tried just replacing those files but had to manually convert the TS one.

Still saw the same error as before when I am not logged in:

~/s/firebase-emulator-test (master|✓) [1] $ yarn emulators --debug
yarn run v1.17.3
$ yarn firebase emulators:start --debug
$ /home/filipesilva/sandbox/firebase-emulator-test/node_modules/.bin/firebase emulators:start --debug
[2019-12-13T18:14:19.558Z] ----------------------------------------------------------------------
[2019-12-13T18:14:19.561Z] Command:       /home/filipesilva/.config/nvm/13.3.0/bin/node /home/filipesilva/sandbox/firebase-emulator-test/node_modules/.bin/firebase emulators:start --debug
[2019-12-13T18:14:19.561Z] CLI Version:   7.9.0
[2019-12-13T18:14:19.561Z] Platform:      linux
[2019-12-13T18:14:19.561Z] Node Version:  v13.3.0
[2019-12-13T18:14:19.562Z] Time:          Fri Dec 13 2019 18:14:19 GMT+0000 (Greenwich Mean Time)
[2019-12-13T18:14:19.562Z] ----------------------------------------------------------------------
[2019-12-13T18:14:19.562Z] 
[2019-12-13T18:14:19.582Z] > command requires scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","https://www.googleapis.com/auth/cloud-platform"]
[2019-12-13T18:14:19.582Z] > attempting to authenticate via app default credentials
[2019-12-13T18:14:19.691Z] ! auto-auth error: Could not load the default credentials. Browse to https://developers.google.com/accounts/docs/application-default-credentials for more information.
[2019-12-13T18:14:19.691Z] > no credentials could be found or automatically retrieved

Error: Command requires authentication, please run firebase login

Having trouble? Try firebase [command] --help
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

[samtstern]

@filipesilva if you want to test the CLI on my branch you can do this (with npm, not sure how yarn works):

# Install locally as a dev dependency
npm install --save-dev firebase/firebase-tools#ss-serve-offline

# Run the local version
npx firebase emulators:start --only hosting

[filipesilva]

@samtstern tried following those instructions and got the same message:

kamik@RED-X1C6 MINGW64 /d/sandbox/firebase-emulator (master)
$ npx firebase emulators:start --only hosting --debug
[2019-12-13T18:32:02.239Z] ----------------------------------------------------------------------
[2019-12-13T18:32:02.243Z] Command:       C:\Program Files\nodejs\node.exe D:\sandbox\firebase-emulator\node_modules\firebase-tools\lib\bin\firebase.js emulators:start --only hosting --debug
[2019-12-13T18:32:02.245Z] CLI Version:   7.10.0
[2019-12-13T18:32:02.246Z] Platform:      win32
[2019-12-13T18:32:02.247Z] Node Version:  v10.16.0
[2019-12-13T18:32:02.250Z] Time:          Fri Dec 13 2019 18:32:02 GMT+0000 (Greenwich Mean Time)
[2019-12-13T18:32:02.266Z] ----------------------------------------------------------------------
[2019-12-13T18:32:02.267Z]
[2019-12-13T18:32:02.281Z] > command requires scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","https://www.googleapis.com/auth/cloud-platform"]
[2019-12-13T18:32:02.283Z] > attempting to authenticate via app default credentials
[2019-12-13T18:32:02.329Z] ! auto-auth error: Could not load the default credentials. Browse to https://developers.google.com/accounts/docs/application-default-credentials for more information.
[2019-12-13T18:32:02.331Z] > no credentials could be found or automatically retrieved

Error: Command requires authentication, please run firebase login

I can verify that I am using https://github.com/firebase/firebase-tools/pull/1864 because node_modules/firebase-tools/lib/hosting/implicitInit.js contains some of the code you added:

        if (!config) {
            config = getCachedWebSetup(options);
            if (config) {
                utils.logLabeledWarning("hosting", "Using web app configuration from cache.");
            }
        }

[ultraGentle]

@samtstern Yes, adding this to emulators:start instead of serve would work for me. Much appreciated!

[samtstern]

@filipesilva thanks! I realized I still had some gcloud application default credentials on my machine. I've updated the PR to fix what you're running into, care to try it again?

[filipesilva]

Tried again now by removing node_modules, reinstalling, and checking the install contained code from your PR.

Then I ran firebase logout before running the commands.

I can confirm it works:

kamik@RED-X1C6 MINGW64 /d/sandbox/firebase-emulator (master)
$ npx firebase emulators:start --only hosting --debug
[2019-12-13T21:40:13.233Z] ----------------------------------------------------------------------
[2019-12-13T21:40:13.237Z] Command:       C:\Program Files\nodejs\node.exe D:\sandbox\firebase-emulator\node_modules\firebase-tools\lib\bin\firebase.js emulators:start --only hosting --debug
[2019-12-13T21:40:13.257Z] CLI Version:   7.10.0
[2019-12-13T21:40:13.258Z] Platform:      win32
[2019-12-13T21:40:13.259Z] Node Version:  v10.16.0
[2019-12-13T21:40:13.262Z] Time:          Fri Dec 13 2019 21:40:13 GMT+0000 (Greenwich Mean Time)
[2019-12-13T21:40:13.263Z] ----------------------------------------------------------------------
[2019-12-13T21:40:13.264Z]
[2019-12-13T21:40:13.281Z] > command requires scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","https://www.googleapis.com/auth/cloud-platform"]
[2019-12-13T21:40:13.290Z] > attempting to authenticate via app default credentials
i  emulators: Starting emulators: hosting
[2019-12-13T21:40:13.341Z] ! auto-auth error: Could not load the default credentials. Browse to https://developers.google.com/accounts/docs/application-default-credentials for more information.
[2019-12-13T21:40:13.347Z] > no credentials could be found or automatically retrieved
(node:20832) UnhandledPromiseRejectionWarning: FirebaseError: Command requires authentication, please run firebase login
    at Object.<anonymous> (D:\sandbox\firebase-emulator\node_modules\firebase-tools\lib\requireAuth.js:10:18)
    at Module._compile (internal/modules/cjs/loader.js:776:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Module.require (internal/modules/cjs/loader.js:690:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Object.<anonymous> (D:\sandbox\firebase-emulator\node_modules\firebase-tools\lib\commands\appdistribution-distribute.js:14:21)
    at Module._compile (internal/modules/cjs/loader.js:776:30)
(node:20832) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:20832) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a 
non-zero exit code.
[2019-12-13T21:40:15.349Z] > refreshing access token with scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","https://www.googleapis.com/auth/cloud-platform"]
[2019-12-13T21:40:15.351Z] >>> HTTP REQUEST POST https://www.googleapis.com/oauth2/v3/token  
 <request body omitted>
[2019-12-13T21:40:15.640Z] <<< HTTP RESPONSE 400 content-type=application/json; charset=utf-8, vary=X-Origin, Referer, Origin,Accept-Encoding, date=Fri, 13 Dec 2019 21:40:15 GMT, server=scaffolding on HTTPServer2, cache-control=private, x-xss-protection=0, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, alt-svc=quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000, accept-ranges=none, transfer-encoding=chunked
[2019-12-13T21:40:15.643Z] >>> HTTP REQUEST GET https://firebase.googleapis.com/v1beta1/projects/emulator-test-1/webApps/-/config  

[2019-12-13T21:40:15.840Z] <<< HTTP RESPONSE 401 www-authenticate=Bearer realm="https://accounts.google.com/", error="invalid_token", vary=X-Origin, Referer, Origin,Accept-Encoding, content-type=application/json; charset=UTF-8, date=Fri, 13 Dec 2019 21:40:16 GMT, server=ESF, cache-control=private, x-xss-protection=0, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, alt-svc=quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000, accept-ranges=none, transfer-encoding=chunked
[2019-12-13T21:40:15.842Z] <<< HTTP RESPONSE BODY code=401, message=Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project., status=UNAUTHENTICATED
[2019-12-13T21:40:15.865Z] fetchWebSetup error: FirebaseError: HTTP Error: 401, Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
!  hosting: Could not fetch web app configuration and there is no cached configuration on this machine. Check your internet connection and make sure you are authenticated. To continue, 
you must call firebase.initializeApp({...}) in your code before using Firebase.
i  hosting: Serving hosting files from: public
+  hosting: Local server: http://localhost:5000
+  hosting: Emulator started at http://localhost:5000
+  All emulators started, it is now safe to connect.

There seems to be an uncaught error that shows up even without debugging being turned on:

kamik@RED-X1C6 MINGW64 /d/sandbox/firebase-emulator (master)
$ npx firebase emulators:start --only hosting 
i  emulators: Starting emulators: hosting
(node:4720) UnhandledPromiseRejectionWarning: FirebaseError: Command requires authentication, please run firebase login
    at Object.<anonymous> (D:\sandbox\firebase-emulator\node_modules\firebase-tools\lib\requireAuth.js:10:18)
    at Module._compile (internal/modules/cjs/loader.js:776:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Module.require (internal/modules/cjs/loader.js:690:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Object.<anonymous> (D:\sandbox\firebase-emulator\node_modules\firebase-tools\lib\commands\appdistribution-distribute.js:14:21)
    at Module._compile (internal/modules/cjs/loader.js:776:30)
(node:4720) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a 
promise which was not handled with .catch(). (rejection id: 1)
(node:4720) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
!  hosting: Could not fetch web app configuration and there is no cached configuration on this machine. Check your internet connection and make sure you are authenticated. To continue, 
you must call firebase.initializeApp({...}) in your code before using Firebase.
i  hosting: Serving hosting files from: public
+  hosting: Local server: http://localhost:5000
+  hosting: Emulator started at http://localhost:5000
+  All emulators started, it is now safe to connect.

While unsightly, it does not seem to affect the functioning of the emulator. I was able to serve all the emulators in https://github.com/filipesilva/firebase-emulator-test via npx firebase emulators:start and see the desired behaviour of realtime database, firestore, and hosting.

I left a comment on the PR as to the light I think might be the problem.

[filipesilva]

I also tried after logging in, but in a codebase where I do not have permissions:

i  emulators: Starting emulators: hosting
!  hosting: Authentication error when trying to fetch web configuration, have you run firebase login?
!  hosting: Could not fetch web app configuration and there is no cached configuration on this machine. Check your internet connection and make sure you are authenticated. To continue, 
you must call firebase.initializeApp({...}) in your code before using Firebase.
i  hosting: Serving hosting files from: resources/public
+  hosting: Local server: http://localhost:5000
+  hosting: Emulator started at http://localhost:5000
+  All emulators started, it is now safe to connect.

[filipesilva]

For the usecases I tested, I think https://github.com/firebase/firebase-tools/pull/1864 is right as rain 🎉

[samtstern]

1864 has been merged and will be included in the next release. Thank you all for your input and helping us get this right!

[filipesilva]

Awesome, thank you so much!

[ultraGentle]

Much appreciated!