search

googleapis / nodejs-firestore

Node.js client for Google Cloud Firestore: a NoSQL document database built for automatic scaling, high performance, and ease of application development.
https://cloud.google.com/firestore/
Apache License 2.0
643 stars 149 forks source link

Firestore SDK still requires credentials even if the target is the emulator #998

Closed samtstern closed 4 years ago

samtstern commented 4 years ago

Environment details

Steps to reproduce

  1. Set FIRESTORE_EMULATOR_HOST=localhost:8080

  2. Attempt to use the Firebase Admin SDK to talk to Firestore in an environment without any Google Cloud credentials.

    It should understand that in this environment there are no credentials needed, but instead there is still some gate when initializing the client,

samtstern commented 4 years ago

@yuchenshi advised me to file this bug.

bcoe commented 4 years ago

@samtstern were you able to find a workaround, or is this still an issue?

samtstern commented 4 years ago

This is still an issue, I first ran into it yesterday.

The workaround is to export GOOGLE_APPLICATION_CREDENTIALS but I would prefer not to do that in my CI environment because it puts me at risk of doing a dangerous operation on prod with a coding error.

schmidt-sebastian commented 4 years ago

I will take a look at this soon.

explunit commented 4 years ago

I'm encountering this problem too, but I see it as a regression between version 3.7.1 and version 3.7.2.

In version 3.7.1 code like this would work (see stackoverflow question):

const {Firestore} = require('@google-cloud/firestore');
const {credentials} = require('@grpc/grpc-js');

const db = new Firestore({
  projectId: 'my-project-id',
  servicePath: 'localhost',
  port: 8100,
  sslCreds: credentials.createInsecure(),
  customHeaders: {
    "Authorization": "Bearer owner"
  }
});

async function load_data() {
  await db.collection("mycollection").doc("myid").set({ foo: "test" })
}
load_data();

But in 3.7.2 (and later), I get this error:

(node:10564) UnhandledPromiseRejectionWarning: TypeError: Channel credentials must be a ChannelCredentials object
    at new ChannelImplementation (C:\GH\close-dirt-web\node_modules\google-gax\node_modules\@grpc\grpc-js\build\src\channel.js:63:19)
    at new Client (C:\GH\close-dirt-web\node_modules\google-gax\node_modules\@grpc\grpc-js\build\src\client.js:56:36)
    at new ServiceClientImpl (C:\GH\close-dirt-web\node_modules\google-gax\node_modules\@grpc\grpc-js\build\src\make-client.js:49:5)
    at GrpcClient.createStub (C:\GH\close-dirt-web\node_modules\google-gax\build\src\grpc.js:220:22)
    at process._tickCallback (internal/process/next_tick.js:68:7)
    at Function.Module.runMain (internal/modules/cjs/loader.js:834:11)
    at startup (internal/bootstrap/node.js:283:19)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:623:3)
schmidt-sebastian commented 4 years ago

@explunit Do you have an explicit dependency on @grpc/grpc-js? If so, you need to ensure that you are using the same version that @google-cloud/firestore uses in order for our internal instanceof checks to work. Firestore 3.7.2 just upgarded to the 0.7.x @grpc/grpc-js train.

explunit commented 4 years ago

@schmidt-sebastian Yes, I updated to @grpc/grpc-js 0.7.6 and received the same error

schmidt-sebastian commented 4 years ago

Can you share your output from npm list?

explunit commented 4 years ago

Yes, there is the npm list output for the failed set of versions:

explunit-firestore-test@1.0.0 C:\Users\SP\Documents\firestoretest
+-- @google-cloud/firestore@3.7.3
| +-- deep-equal@2.0.2
| | +-- es-abstract@1.17.5
| | | +-- es-to-primitive@1.2.1
| | | | +-- is-callable@1.1.5 deduped
| | | | +-- is-date-object@1.0.2 deduped
| | | | `-- is-symbol@1.0.3 deduped
| | | +-- function-bind@1.1.1
| | | +-- has@1.0.3
| | | | `-- function-bind@1.1.1 deduped
| | | +-- has-symbols@1.0.1
| | | +-- is-callable@1.1.5
| | | +-- is-regex@1.0.5 deduped
| | | +-- object-inspect@1.7.0
| | | +-- object-keys@1.1.1 deduped
| | | +-- object.assign@4.1.0
| | | | +-- define-properties@1.1.3 deduped
| | | | +-- function-bind@1.1.1 deduped
| | | | +-- has-symbols@1.0.1 deduped
| | | | `-- object-keys@1.1.1 deduped
| | | +-- string.prototype.trimleft@2.1.2
| | | | +-- define-properties@1.1.3 deduped
| | | | +-- es-abstract@1.17.5 deduped
| | | | `-- string.prototype.trimstart@1.0.0
| | | |   +-- define-properties@1.1.3 deduped
| | | |   `-- es-abstract@1.17.5 deduped
| | | `-- string.prototype.trimright@2.1.2
| | |   +-- define-properties@1.1.3 deduped
| | |   +-- es-abstract@1.17.5 deduped
| | |   `-- string.prototype.trimend@1.0.0
| | |     +-- define-properties@1.1.3 deduped
| | |     `-- es-abstract@1.17.5 deduped
| | +-- es-get-iterator@1.1.0
| | | +-- es-abstract@1.17.5 deduped
| | | +-- has-symbols@1.0.1 deduped
| | | +-- is-arguments@1.0.4 deduped
| | | +-- is-map@2.0.1
| | | +-- is-set@2.0.1
| | | +-- is-string@1.0.5
| | | `-- isarray@2.0.5 deduped
| | +-- is-arguments@1.0.4
| | +-- is-date-object@1.0.2
| | +-- is-regex@1.0.5
| | | `-- has@1.0.3 deduped
| | +-- isarray@2.0.5
| | +-- object-is@1.0.2
| | +-- object-keys@1.1.1
| | +-- regexp.prototype.flags@1.3.0
| | | +-- define-properties@1.1.3
| | | | `-- object-keys@1.1.1 deduped
| | | `-- es-abstract@1.17.5 deduped
| | +-- side-channel@1.0.2
| | | +-- es-abstract@1.17.5 deduped
| | | `-- object-inspect@1.7.0 deduped
| | +-- which-boxed-primitive@1.0.1
| | | +-- is-bigint@1.0.0
| | | +-- is-boolean-object@1.0.1
| | | +-- is-number-object@1.0.4
| | | +-- is-string@1.0.5 deduped
| | | `-- is-symbol@1.0.3
| | |   `-- has-symbols@1.0.1 deduped
| | +-- which-collection@1.0.1
| | | +-- is-map@2.0.1 deduped
| | | +-- is-set@2.0.1 deduped
| | | +-- is-weakmap@2.0.1
| | | `-- is-weakset@2.0.1
| | `-- which-typed-array@1.1.1
| |   +-- available-typed-arrays@1.0.2
| |   | `-- array-filter@1.0.0
| |   +-- es-abstract@1.17.5 deduped
| |   +-- function-bind@1.1.1 deduped
| |   +-- has-symbols@1.0.1 deduped
| |   `-- is-typed-array@1.1.3
| |     +-- available-typed-arrays@1.0.2 deduped
| |     +-- es-abstract@1.17.5 deduped
| |     +-- foreach@2.0.5
| |     `-- has-symbols@1.0.1 deduped
| +-- functional-red-black-tree@1.0.1
| +-- google-gax@1.15.2
| | +-- @grpc/grpc-js@0.7.6
| | | `-- semver@6.3.0 deduped
| | +-- @grpc/proto-loader@0.5.4
| | | +-- lodash.camelcase@4.3.0
| | | `-- protobufjs@6.8.9 deduped
| | +-- @types/fs-extra@8.1.0
| | | `-- @types/node@10.17.18
| | +-- @types/long@4.0.1
| | +-- abort-controller@3.0.0
| | | `-- event-target-shim@5.0.1
| | +-- duplexify@3.7.1
| | | +-- end-of-stream@1.4.4
| | | | `-- once@1.4.0
| | | |   `-- wrappy@1.0.2
| | | +-- inherits@2.0.4 deduped
| | | +-- readable-stream@2.3.7
| | | | +-- core-util-is@1.0.2
| | | | +-- inherits@2.0.4 deduped
| | | | +-- isarray@1.0.0
| | | | +-- process-nextick-args@2.0.1
| | | | +-- safe-buffer@5.1.2 deduped
| | | | +-- string_decoder@1.1.1 deduped
| | | | `-- util-deprecate@1.0.2 deduped
| | | `-- stream-shift@1.0.1
| | +-- google-auth-library@5.10.1
| | | +-- arrify@2.0.1
| | | +-- base64-js@1.3.1
| | | +-- ecdsa-sig-formatter@1.0.11
| | | | `-- safe-buffer@5.1.2 deduped
| | | +-- fast-text-encoding@1.0.1
| | | +-- gaxios@2.3.4
| | | | +-- abort-controller@3.0.0 deduped
| | | | +-- extend@3.0.2
| | | | +-- https-proxy-agent@5.0.0
| | | | | +-- agent-base@6.0.0
| | | | | | `-- debug@4.1.1 deduped
| | | | | `-- debug@4.1.1 deduped
| | | | +-- is-stream@2.0.0
| | | | `-- node-fetch@2.6.0 deduped
| | | +-- gcp-metadata@3.5.0
| | | | +-- gaxios@2.3.4 deduped
| | | | `-- json-bigint@0.3.0
| | | |   `-- bignumber.js@7.2.1
| | | +-- gtoken@4.1.4
| | | | +-- gaxios@2.3.4 deduped
| | | | +-- google-p12-pem@2.0.4
| | | | | `-- node-forge@0.9.1
| | | | +-- jws@4.0.0 deduped
| | | | `-- mime@2.4.4
| | | +-- jws@4.0.0
| | | | +-- jwa@2.0.0
| | | | | +-- buffer-equal-constant-time@1.0.1
| | | | | +-- ecdsa-sig-formatter@1.0.11 deduped
| | | | | `-- safe-buffer@5.1.2 deduped
| | | | `-- safe-buffer@5.1.2 deduped
| | | `-- lru-cache@5.1.1
| | |   `-- yallist@3.1.1
| | +-- is-stream-ended@0.1.4
| | +-- lodash.at@4.6.0
| | +-- lodash.has@4.5.2
| | +-- node-fetch@2.6.0
| | +-- protobufjs@6.8.9
| | | +-- @protobufjs/aspromise@1.1.2
| | | +-- @protobufjs/base64@1.1.2
| | | +-- @protobufjs/codegen@2.0.4
| | | +-- @protobufjs/eventemitter@1.1.0
| | | +-- @protobufjs/fetch@1.1.0
| | | | +-- @protobufjs/aspromise@1.1.2 deduped
| | | | `-- @protobufjs/inquire@1.1.0 deduped
| | | +-- @protobufjs/float@1.0.2
| | | +-- @protobufjs/inquire@1.1.0
| | | +-- @protobufjs/path@1.1.2
| | | +-- @protobufjs/pool@1.1.0
| | | +-- @protobufjs/utf8@1.1.0
| | | +-- @types/long@4.0.1 deduped
| | | +-- @types/node@10.17.18 deduped
| | | `-- long@4.0.0
| | +-- retry-request@4.1.1
| | | +-- debug@4.1.1
| | | | `-- ms@2.1.2
| | | `-- through2@3.0.1 deduped
| | +-- semver@6.3.0 deduped
| | `-- walkdir@0.4.1
| +-- readable-stream@3.6.0
| | +-- inherits@2.0.4
| | +-- string_decoder@1.1.1
| | | `-- safe-buffer@5.1.2
| | `-- util-deprecate@1.0.2
| `-- through2@3.0.1
|   `-- readable-stream@3.6.0 deduped
`-- @grpc/grpc-js@0.7.6
  `-- semver@6.3.0

And here is the npm list output for the working set of versions:

explunit-firestore-test@1.0.0 C:\Users\SP\Documents\firestoretest
+-- @google-cloud/firestore@3.7.1
| +-- deep-equal@2.0.2
| | +-- es-abstract@1.17.5
| | | +-- es-to-primitive@1.2.1
| | | | +-- is-callable@1.1.5 deduped
| | | | +-- is-date-object@1.0.2 deduped
| | | | `-- is-symbol@1.0.3 deduped
| | | +-- function-bind@1.1.1
| | | +-- has@1.0.3
| | | | `-- function-bind@1.1.1 deduped
| | | +-- has-symbols@1.0.1
| | | +-- is-callable@1.1.5
| | | +-- is-regex@1.0.5 deduped
| | | +-- object-inspect@1.7.0
| | | +-- object-keys@1.1.1 deduped
| | | +-- object.assign@4.1.0
| | | | +-- define-properties@1.1.3 deduped
| | | | +-- function-bind@1.1.1 deduped
| | | | +-- has-symbols@1.0.1 deduped
| | | | `-- object-keys@1.1.1 deduped
| | | +-- string.prototype.trimleft@2.1.2
| | | | +-- define-properties@1.1.3 deduped
| | | | +-- es-abstract@1.17.5 deduped
| | | | `-- string.prototype.trimstart@1.0.0
| | | |   +-- define-properties@1.1.3 deduped
| | | |   `-- es-abstract@1.17.5 deduped
| | | `-- string.prototype.trimright@2.1.2
| | |   +-- define-properties@1.1.3 deduped
| | |   +-- es-abstract@1.17.5 deduped
| | |   `-- string.prototype.trimend@1.0.0
| | |     +-- define-properties@1.1.3 deduped
| | |     `-- es-abstract@1.17.5 deduped
| | +-- es-get-iterator@1.1.0
| | | +-- es-abstract@1.17.5 deduped
| | | +-- has-symbols@1.0.1 deduped
| | | +-- is-arguments@1.0.4 deduped
| | | +-- is-map@2.0.1
| | | +-- is-set@2.0.1
| | | +-- is-string@1.0.5
| | | `-- isarray@2.0.5
| | +-- is-arguments@1.0.4
| | +-- is-date-object@1.0.2
| | +-- is-regex@1.0.5
| | | `-- has@1.0.3 deduped
| | +-- isarray@2.0.5
| | +-- object-is@1.0.2
| | +-- object-keys@1.1.1
| | +-- regexp.prototype.flags@1.3.0
| | | +-- define-properties@1.1.3
| | | | `-- object-keys@1.1.1 deduped
| | | `-- es-abstract@1.17.5 deduped
| | +-- side-channel@1.0.2
| | | +-- es-abstract@1.17.5 deduped
| | | `-- object-inspect@1.7.0 deduped
| | +-- which-boxed-primitive@1.0.1
| | | +-- is-bigint@1.0.0
| | | +-- is-boolean-object@1.0.1
| | | +-- is-number-object@1.0.4
| | | +-- is-string@1.0.5 deduped
| | | `-- is-symbol@1.0.3
| | |   `-- has-symbols@1.0.1 deduped
| | +-- which-collection@1.0.1
| | | +-- is-map@2.0.1 deduped
| | | +-- is-set@2.0.1 deduped
| | | +-- is-weakmap@2.0.1
| | | `-- is-weakset@2.0.1
| | `-- which-typed-array@1.1.1
| |   +-- available-typed-arrays@1.0.2
| |   | `-- array-filter@1.0.0
| |   +-- es-abstract@1.17.5 deduped
| |   +-- function-bind@1.1.1 deduped
| |   +-- has-symbols@1.0.1 deduped
| |   `-- is-typed-array@1.1.3
| |     +-- available-typed-arrays@1.0.2 deduped
| |     +-- es-abstract@1.17.5 deduped
| |     +-- foreach@2.0.5
| |     `-- has-symbols@1.0.1 deduped
| +-- functional-red-black-tree@1.0.1
| +-- google-gax@1.15.1
| | +-- @grpc/grpc-js@0.6.18 deduped
| | +-- @grpc/proto-loader@0.5.4
| | | +-- lodash.camelcase@4.3.0
| | | `-- protobufjs@6.8.9 deduped
| | +-- @types/fs-extra@8.1.0
| | | `-- @types/node@10.17.18
| | +-- @types/long@4.0.1
| | +-- abort-controller@3.0.0
| | | `-- event-target-shim@5.0.1
| | +-- duplexify@3.7.1
| | | +-- end-of-stream@1.4.4
| | | | `-- once@1.4.0
| | | |   `-- wrappy@1.0.2
| | | +-- inherits@2.0.4 deduped
| | | +-- readable-stream@2.3.7 deduped
| | | `-- stream-shift@1.0.1
| | +-- google-auth-library@5.10.1
| | | +-- arrify@2.0.1
| | | +-- base64-js@1.3.1
| | | +-- ecdsa-sig-formatter@1.0.11
| | | | `-- safe-buffer@5.1.2 deduped
| | | +-- fast-text-encoding@1.0.1
| | | +-- gaxios@2.3.4
| | | | +-- abort-controller@3.0.0 deduped
| | | | +-- extend@3.0.2
| | | | +-- https-proxy-agent@5.0.0
| | | | | +-- agent-base@6.0.0
| | | | | | `-- debug@4.1.1 deduped
| | | | | `-- debug@4.1.1 deduped
| | | | +-- is-stream@2.0.0
| | | | `-- node-fetch@2.6.0 deduped
| | | +-- gcp-metadata@3.5.0
| | | | +-- gaxios@2.3.4 deduped
| | | | `-- json-bigint@0.3.0
| | | |   `-- bignumber.js@7.2.1
| | | +-- gtoken@4.1.4
| | | | +-- gaxios@2.3.4 deduped
| | | | +-- google-p12-pem@2.0.4
| | | | | `-- node-forge@0.9.1
| | | | +-- jws@4.0.0 deduped
| | | | `-- mime@2.4.4
| | | +-- jws@4.0.0
| | | | +-- jwa@2.0.0
| | | | | +-- buffer-equal-constant-time@1.0.1
| | | | | +-- ecdsa-sig-formatter@1.0.11 deduped
| | | | | `-- safe-buffer@5.1.2 deduped
| | | | `-- safe-buffer@5.1.2 deduped
| | | `-- lru-cache@5.1.1
| | |   `-- yallist@3.1.1
| | +-- is-stream-ended@0.1.4
| | +-- lodash.at@4.6.0
| | +-- lodash.has@4.5.2
| | +-- node-fetch@2.6.0
| | +-- protobufjs@6.8.9
| | | +-- @protobufjs/aspromise@1.1.2
| | | +-- @protobufjs/base64@1.1.2
| | | +-- @protobufjs/codegen@2.0.4
| | | +-- @protobufjs/eventemitter@1.1.0
| | | +-- @protobufjs/fetch@1.1.0
| | | | +-- @protobufjs/aspromise@1.1.2 deduped
| | | | `-- @protobufjs/inquire@1.1.0 deduped
| | | +-- @protobufjs/float@1.0.2
| | | +-- @protobufjs/inquire@1.1.0
| | | +-- @protobufjs/path@1.1.2
| | | +-- @protobufjs/pool@1.1.0
| | | +-- @protobufjs/utf8@1.1.0
| | | +-- @types/long@4.0.1 deduped
| | | +-- @types/node@10.17.18 deduped
| | | `-- long@4.0.0
| | +-- retry-request@4.1.1
| | | +-- debug@4.1.1
| | | | `-- ms@2.1.2
| | | `-- through2@3.0.1 deduped
| | +-- semver@6.3.0 deduped
| | `-- walkdir@0.4.1
| +-- readable-stream@3.6.0
| | +-- inherits@2.0.4
| | +-- string_decoder@1.1.1
| | | `-- safe-buffer@5.1.2
| | `-- util-deprecate@1.0.2
| `-- through2@3.0.1
|   `-- readable-stream@2.3.7
|     +-- core-util-is@1.0.2
|     +-- inherits@2.0.4 deduped
|     +-- isarray@1.0.0
|     +-- process-nextick-args@2.0.1
|     +-- safe-buffer@5.1.2 deduped
|     +-- string_decoder@1.1.1 deduped
|     `-- util-deprecate@1.0.2 deduped
`-- @grpc/grpc-js@0.6.18
  `-- semver@6.3.0
schmidt-sebastian commented 4 years ago

@explunit You are using two versions of @grpc/grpc-js:

| | +-- @grpc/grpc-js@0.7.6
`-- @grpc/grpc-js@0.7.6
| | +-- @grpc/grpc-js@0.6.18 deduped
`-- @grpc/grpc-js@0.6.18

Passing 0.6.18 ChannelCredentials to 0.7.6 will not work. You should either remove your explicit dependency on @grpc/grpc-js or upgrade it to match the version used by @google-cloud/firestore.

explunit commented 4 years ago

@schmidt-sebastian No, I'm not using two versions. Note that I posted two different versions of npm list -- the broken one (post-upgrade) and the working one (pre-upgrade).

schmidt-sebastian commented 4 years ago

Sorry, I misread your output and combined the two. I looked through the sources and cannot explain the behavior you are seeing.

Upon investigating the root cause for this issue, though, it seems like simply setting GCLOUD_PROJECT should let you connect to the emulator without specifying your own credentials. Does this unblock you?

In the meantime, I am figuring out if we can remove the requirement to set GCLOUD_PROJECT.

explunit commented 4 years ago

I'm willing to try that but I'm not sure exactly what you're suggesting I change in my sample code.

samtstern commented 4 years ago

@explunit you want to export the GCLOUD_PROJECT environment variable and set it to your project ID. 

export GCLOUD_PROJECT=my-project-1234
export FIRESTORE_EMULATOR_HOST=localhost:8080
samtstern commented 4 years ago

Thanks Sebastian for digging. Looks like this is working as intended for now and we just need to update the docs to make sure people set both variables.

explunit commented 4 years ago

Thanks. I confirm it works for me with both those environment variables set. Then the sample code is much simpler:

const {Firestore} = require('@google-cloud/firestore');

const db = new Firestore();

async function load_data() {
  await db.collection("mycollection").doc("myid").set({ foo: "test" })
}
load_data();
elucidsoft commented 4 years ago

I have two project on my machine, one is working fine the other is not. One the first one I do not set these environment variables at all and it just works. On the second one, its one I just created and I can't for the life of me figure out why I get this error as the package.json versions match exactly in both projects.

Update: I can confirm after removing node_modules and deleting package-lock.json and re-running npm i that this is happening. Setting the environment variables has no effect for me at all. Same error.

jirizavadil commented 4 years ago

@elucidsoft check if you're not a dumb f*** like me and installed 'firebase' module, which is frontend only. spent 6 hours on it.

emalamisura-c2m commented 4 years ago

I think the issue is that I use Firestore from firebase-admin because I also use firebase-auth.

emalamisura-c2m commented 4 years ago

Solved: The solution is pretty stupid tbh, as @jirizavadil mentioned I had firebase installed somehow, removing that didn't fix anything. I also had @grpc/grpc-js and grpc installed as devDependencies for some reason, not sure how those got there. Removing those fixed nothing. I removed node_modules dir and did an npm i, fixed nothing. I removed node_modules, and package-lock.json and did an npm i. Fixed! Mind you this was from a completely working state, all I did was run npm update, and had to go through this process.

arri-cc commented 4 years ago

we had all of these issues to resolve. It now works after:

  1. setting GCLOUD_PROJECT environment variable
  2. setting FIREBASE_EMULATOR_HOST environment variable
  3. removing the firebase npm package from our functions package
  4. Deleting the node_modules folder and package-lock.json and reinstalling the npm packages.

now using the firestore emulator from the cloud functions emulator works as expected.

nwparker commented 4 years ago

I'm also running into this issue, but none of the above steps mentioned resolve my issues.

In my case, I'm following advice from https://github.com/firebase/firebase-admin-node/issues/575#issuecomment-524744793 to connect my admin SDK (Firestore) to the emulator and am seeing the issue.

As other have stated, this has only started happening recently (after updating npm packages)

vongohren commented 4 years ago

Im running my firebase admin inside a next.js project. So server side rendering. But I cannot remove the firebase, as I use it for my client side auth. And Im using firebase admin for my requests coming from the client, handled intrinsicly by next.js so not a clear API code.

So what I read here is that firebase module creates crashes? But how can we get this to work without removing that module?

I have set the two env variables as well

Im seeing this issue with the newest versions of both libs

Update

Just to follow up on this, this comment has never been discussed: https://stackoverflow.com/a/61726794/2488692

But doing that worked for me

In short $ npm i -D google-gax

My solution

  1. setting FIREBASE_EMULATOR_HOST environment variable
  2. Updating the libs, not sure if that is really needed
  3. Installing google-gax
manwithsteelnerves commented 4 years ago

I didn't install google-gax but just set FIREBASE_EMULATOR_HOST to localhost:8080 and it worked!