search

firebase / firebase-tools

The Firebase Command Line Tools
MIT License
4.04k stars 952 forks source link

Emulator increasing used memory and lose the connection with the Firestore #4916

Closed anderjf closed 2 years ago

anderjf commented 2 years ago

[REQUIRED] Environment info

firebase-tools: 11.7.0

Platform: Ubuntu 20.04.4 LTS (5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux)

JVM: Java(TM) SE Runtime Environment (build 17.0.4+11-LTS-179)

Computer: Acer NITRO5, i7, 16GB RAM, SSD 256GB.

[REQUIRED] Test case

When I use a database exported from the production (the files exported from the production have 652MB). I import this backup when Firebase Emulator is starting with the parameter --import, when I am retrieving all documents (about 200.000) from a collection, after some seconds show the error message Error: 14 UNAVAILABLE: No connection established and the Emulator receives the signal to close Firestore Emulator has exited upon receiving signal: SIGKILL.

[REQUIRED] Steps to reproduce

export JAVA_TOOL_OPTIONS="-Xmx8g -Xms8g"

firebase emulators:start --project demo-backup --import ./backup

const path = '/collection'; const account = db.collection(path);

let count = 0;

account.stream().on('data', (documentSnapshot) => { console.log(documentSnapshot.ref.path); ++count; }).on('end', () => { console.log(Total count is ${count}); });

[REQUIRED] Expected behavior

I expect the code runs until shows the total.

[REQUIRED] Actual behavior

Error in the terminal

node:events:491 throw er; // Unhandled 'error' event ^

Error: 14 UNAVAILABLE: No connection established at Object.callErrorFromStatus (.../node_modules/@grpc/grpc-js/build/src/call.js:31:19) at Object.onReceiveStatus (.../node_modules/@grpc/grpc-js/build/src/client.js:352:49) at Object.onReceiveStatus (.../node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181) at .../node_modules/@grpc/grpc-js/build/src/call-stream.js:188:78 at processTicksAndRejections (node:internal/process/task_queues:78:11) for call at at ServiceClientImpl.makeServerStreamRequest (.../node_modules/@grpc/grpc-js/build/src/client.js:336:30) at ServiceClientImpl. (.../node_modules/@grpc/grpc-js/build/src/make-client.js:105:19) at .../node_modules/firebase-admin/node_modules/@google-cloud/firestore/build/src/v1/firestore_client.js:206:29 at .../node_modules/firebase-admin/node_modules/google-gax/build/src/streamingCalls/streamingApiCaller.js:38:28 at .../node_modules/firebase-admin/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16 at Object.request (.../node_modules/firebase-admin/node_modules/google-gax/build/src/streamingCalls/streaming.js:126:40) at Timeout.makeRequest [as _onTimeout] (.../node_modules/firebase-admin/node_modules/google-gax/node_modules/retry-request/index.js:139:28) at listOnTimeout (node:internal/timers:559:17) at processTimers (node:internal/timers:502:7) Emitted 'error' event on Transform instance at: at Transform.onerror (node:internal/streams/readable:773:14) at Transform.emit (node:events:513:28) at emitErrorNT (node:internal/streams/destroy:157:8) at emitErrorCloseNT (node:internal/streams/destroy:122:3) at processTicksAndRejections (node:internal/process/task_queues:83:21) { code: 14, details: 'No connection established', metadata: Metadata { internalRepr: Map(0) {}, options: {} } }

Debug log

[debug] [2022-08-23T00:39:15.888Z] ---------------------------------------------------------------------- [debug] [2022-08-23T00:39:15.889Z] Command: /home/.../.nvm/versions/node/v16.17.0/bin/node /home/.../.nvm/versions/node/v16.17.0/bin/firebase emulators:start --project demo-backup --import ./backup [debug] [2022-08-23T00:39:15.890Z] CLI Version: 11.7.0 [debug] [2022-08-23T00:39:15.890Z] Platform: linux [debug] [2022-08-23T00:39:15.890Z] Node Version: v16.17.0 [debug] [2022-08-23T00:39:15.899Z] Time: Tue Aug 23 2022 12:39:15 GMT+1200 (New Zealand Standard Time) [debug] [2022-08-23T00:39:15.900Z] ---------------------------------------------------------------------- [debug] [debug] [2022-08-23T00:39:15.974Z] > command requires scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","https://www.googleapis.com/auth/cloud-platform"] [debug] [2022-08-23T00:39:15.974Z] > authorizing via signed-in user (xxxx@xxxx) [debug] [2022-08-23T00:39:16.033Z] Picked up JAVA_TOOL_OPTIONS: -Xmx8g -Xms8g

[debug] [2022-08-23T00:39:16.253Z] java version "17.0.4" 2022-07-19 LTS

[debug] [2022-08-23T00:39:16.253Z] Java(TM) SE Runtime Environment (build 17.0.4+11-LTS-179) Java HotSpot(TM) 64-Bit Server VM (build 17.0.4+11-LTS-179, mixed mode, sharing)

[debug] [2022-08-23T00:39:16.267Z] Parsed Java major version: 17 [info] i emulators: Starting emulators: firestore {"metadata":{"emulator":{"name":"hub"},"message":"Starting emulators: firestore"}} [info] i emulators: Detected demo project ID "demo-backup", emulated services will use a demo configuration and attempts to access non-emulated services for this project will fail. {"metadata":{"emulator":{"name":"hub"},"message":"Detected demo project ID \"demo-backup\", emulated services will use a demo configuration and attempts to access non-emulated services for this project will fail."}} [debug] [2022-08-23T00:39:16.289Z] [hub] writing locator at /tmp/hub-demo-backup.json [info] i firestore: Detected non-emulator Firestore export at /home/dev/.../backup {"metadata":{"emulator":{"name":"firestore"},"message":"Detected non-emulator Firestore export at /home/dev/.../backup"}} [info] i firestore: Importing data from /home/dev/.../backup/2022-08-11T08:00:04_26595.overall_export_metadata {"metadata":{"emulator":{"name":"firestore"},"message":"Importing data from /home/dev/.../backup/2022-08-11T08:00:04_26595.overall_export_metadata"}} [warn] ⚠ firestore: Did not find a Cloud Firestore rules file specified in a firebase.json config file. {"metadata":{"emulator":{"name":"firestore"},"message":"Did not find a Cloud Firestore rules file specified in a firebase.json config file."}} [warn] ⚠ firestore: The emulator will default to allowing all reads and writes. Learn more about this option: https://firebase.google.com/docs/emulator-suite/install_and_configure#security_rules_configuration. {"metadata":{"emulator":{"name":"firestore"},"message":"The emulator will default to allowing all reads and writes. Learn more about this option: https://firebase.google.com/docs/emulator-suite/install_and_configure#security_rules_configuration."}} [debug] [2022-08-23T00:39:16.309Z] Ignoring unsupported arg: projectId {"metadata":{"emulator":{"name":"firestore"},"message":"Ignoring unsupported arg: projectId"}} [debug] [2022-08-23T00:39:16.309Z] Ignoring unsupported arg: auto_download {"metadata":{"emulator":{"name":"firestore"},"message":"Ignoring unsupported arg: auto_download"}} [debug] [2022-08-23T00:39:16.309Z] Starting Firestore Emulator with command {"binary":"java","args":["-Dgoogle.cloud_firestore.debug_log_level=FINE","-Duser.language=en","-jar","/home/.../.cache/firebase/emulators/cloud-firestore-emulator-v1.14.4.jar","--host","localhost","--port",8180,"--seed_from_export","/home/dev/.../backup/2022-08-11T08:00:04_26595.overall_export_metadata"],"optionalArgs":["port","webchannel_port","host","rules","functions_emulator","seed_from_export"],"joinArgs":false} {"metadata":{"emulator":{"name":"firestore"},"message":"Starting Firestore Emulator with command {\"binary\":\"java\",\"args\":[\"-Dgoogle.cloud_firestore.debug_log_level=FINE\",\"-Duser.language=en\",\"-jar\",\"/home/.../.cache/firebase/emulators/cloud-firestore-emulator-v1.14.4.jar\",\"--host\",\"localhost\",\"--port\",8180,\"--seed_from_export\",\"/home/dev/.../backup/2022-08-11T08:00:04_26595.overall_export_metadata\"],\"optionalArgs\":[\"port\",\"webchannel_port\",\"host\",\"rules\",\"functions_emulator\",\"seed_from_export\"],\"joinArgs\":false}"}} [info] i firestore: Firestore Emulator logging to firestore-debug.log {"metadata":{"emulator":{"name":"firestore"},"message":"Firestore Emulator logging to \u001b[1mfirestore-debug.log\u001b[22m"}} [debug] [2022-08-23T00:39:16.319Z] Picked up JAVA_TOOL_OPTIONS: -Xmx8g -Xms8g {"metadata":{"emulator":{"name":"firestore"},"message":"Picked up JAVA_TOOL_OPTIONS: -Xmx8g -Xms8g\n"}} [debug] [2022-08-23T00:39:23.451Z] Aug 23, 2022 12:39:23 PM com.google.cloud.datastore.emulator.firestore.websocket.WebSocketServer start INFO: Started WebSocket server on ws://localhost:41181 {"metadata":{"emulator":{"name":"firestore"},"message":"Aug 23, 2022 12:39:23 PM com.google.cloud.datastore.emulator.firestore.websocket.WebSocketServer start\nINFO: Started WebSocket server on ws://localhost:41181\n"}} [debug] [2022-08-23T00:39:23.491Z] API endpoint: http://localhost:8180 {"metadata":{"emulator":{"name":"firestore"},"message":"API endpoint: http://localhost:8180\n"}} [debug] [2022-08-23T00:39:23.491Z] If you are using a library that supports the FIRESTORE_EMULATOR_HOST environment variable, run:

export FIRESTORE_EMULATOR_HOST=localhost:8180

Dev App Server is now running.

{"metadata":{"emulator":{"name":"firestore"},"message":"If you are using a library that supports the FIRESTORE_EMULATOR_HOST environment variable, run:\n\n export FIRESTORE_EMULATOR_HOST=localhost:8180\n\nDev App Server is now running.\n\n"}} [debug] [2022-08-23T00:39:23.616Z] Ignoring unsupported arg: auto_download {"metadata":{"emulator":{"name":"ui"},"message":"Ignoring unsupported arg: auto_download"}} [debug] [2022-08-23T00:39:23.616Z] Ignoring unsupported arg: port {"metadata":{"emulator":{"name":"ui"},"message":"Ignoring unsupported arg: port"}} [debug] [2022-08-23T00:39:23.617Z] Starting Emulator UI with command {"binary":"node","args":["/home/.../.cache/firebase/emulators/ui-v1.9.0/server/server.js"],"optionalArgs":[],"joinArgs":false} {"metadata":{"emulator":{"name":"ui"},"message":"Starting Emulator UI with command {\"binary\":\"node\",\"args\":[\"/home/.../.cache/firebase/emulators/ui-v1.9.0/server/server.js\"],\"optionalArgs\":[],\"joinArgs\":false}"}} [info] i ui: Emulator UI logging to ui-debug.log {"metadata":{"emulator":{"name":"ui"},"message":"Emulator UI logging to \u001b[1mui-debug.log\u001b[22m"}} [debug] [2022-08-23T00:39:23.756Z] Web / API server started at localhost:4000 {"metadata":{"emulator":{"name":"ui"},"message":"Web / API server started at localhost:4000\n"}} [info] ┌─────────────────────────────────────────────────────────────┐ │ ✔ All emulators ready! It is now safe to connect your app. │ │ i View Emulator UI at http://localhost:4000 │ └─────────────────────────────────────────────────────────────┘

┌───────────┬────────────────┬─────────────────────────────────┐ │ Emulator │ Host:Port │ View in Emulator UI │ ├───────────┼────────────────┼─────────────────────────────────┤ │ Firestore │ localhost:8180 │ http://localhost:4000/firestore │ └───────────┴────────────────┴─────────────────────────────────┘ Emulator Hub running at localhost:4400 Other reserved ports: 4500

Issues? Report them at https://github.com/firebase/firebase-tools/issues and attach the *-debug.log files.

[debug] [2022-08-23T00:40:51.634Z] Aug 23, 2022 12:40:51 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead INFO: Detected HTTP/2 connection. {"metadata":{"emulator":{"name":"firestore"},"message":"Aug 23, 2022 12:40:51 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead\nINFO: Detected HTTP/2 connection.\n"}} [debug] [2022-08-23T00:44:39.319Z] Aug 23, 2022 12:44:39 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead INFO: Detected HTTP/2 connection. {"metadata":{"emulator":{"name":"firestore"},"message":"Aug 23, 2022 12:44:39 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead\nINFO: Detected HTTP/2 connection.\n"}} [warn] ⚠ Firestore Emulator has exited upon receiving signal: SIGKILL

google-oss-bot commented 2 years ago

This issue does not have all the information required by the template. Looks like you forgot to fill out some sections. Please update the issue with more information.