Firestore emulator sometimes fails on data export when stopped

[madmacc]

Describe your environment:

• Operating System version: Windows 10
• Browser version: Chrome 88.0.4324.104
• Firebase SDK version: 8.0.0
• Firebase Product: Firestore local emulator

Problem: When I stop my local firestore emulator most of the time it exports the data fine. However sometimes is has an error and it deletes my export files so I lose all my data. Nothing seems to appear in the logs. I am using: firebase emulators:start --import=emulator/emulator-export-data --export-on-exit=emulator/emulator-export-data

Then: Ctrl+C to stop the emulator.

I get the following in the terminal:

i emulators: Received SIGINT (Ctrl-C) for the first time. Starting a clean shutdown. i emulators: Please wait for a clean shutdown or send the SIGINT (Ctrl-C) signal again to stop right now. i Automatically exporting data using --export-on-exit "emulator/emulator-export-data" please wait for the export to finish... i Found running emulator hub for project hivepass-dev at http://localhost:4401 i Deleting directory R:\hivepass\HivePassApp\emulator\emulator-export-data\firestore_export i Exporting data to: R:\hivepass\HivePassApp\emulator\emulator-export-data ! Export request failed, see emulator logs for more information. ! Automatic export to "emulator/emulator-export-data" failed, going to exit now... i emulators: Shutting down emulators. i ui: Stopping Emulator UI ! Emulator UI has exited upon receiving signal: SIGINT i hosting: Stopping Hosting Emulator i firestore: Stopping Firestore Emulator ! Firestore Emulator has exited upon receiving signal: SIGINT i auth: Stopping Authentication Emulator i hub: Stopping emulator hub i logging: Stopping Logging Emulator

There is not much in the firestore-debug.log in the root:

INFO: Detected non-HTTP/2 connection. Jan 28, 2021 12:46:37 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead INFO: Detected non-HTTP/2 connection. Jan 28, 2021 12:46:38 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead INFO: Detected non-HTTP/2 connection. Jan 28, 2021 12:46:38 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead INFO: Detected non-HTTP/2 connection. Jan 28, 2021 12:46:39 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead INFO: Detected non-HTTP/2 connection. Jan 28, 2021 12:46:39 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead INFO: Detected non-HTTP/2 connection.

[schmidt-sebastian]

@yuchenshi - Can you take a look at this?

[sam-gc]

Hi @madmacc, thanks for the report.

Would you mind expanding a bit about the steps you're taking before seeing this issue? For example, how much data is being saved? What SDKs are you using? Are you using any of the other emulators in conjunction with Firestore?

[madmacc]

Sure @samhorlbeck. The data is only 130k. I am running the following emulators. Firestore, Auth, Functions However I am running the functions emulator as a separate instance as the functions folder is in a different project/location and I can't figure out how to run all of them with one command.

I am using v9.2.1 of the firebase-tools.

Let me know if you need any more information.

[sam-gc]

What is your general use case? Can you discern any particular action that reliably causes the data to not be saved correctly? Basically I'm just trying to get a sense of how we could start to debug this issue and potentially reproduce it.

[madmacc]

Unfortunately not. I'm sure there is some reason but it seems random to me at this stage. I can run it loading the previously exported data. Add some more data and then stop it by hitting ctrl+c in the terminal. It will sometimes export ok and sometimes not. It only deletes the firestore_export folder not the firebase-export-metadata.json file.

At the moment when it is successful I manually copy the exported data to a safe location so it is not deleted.

I was hoping the logs would show something but they don't unless I am looking in the wrong place. Root of the project firestore-debug.log.

I am just testing an web application locally using the firebase js sdk and angularfire.

[madmacc]

I just tried starting and stopping it about 10 times in a row and it worked every time. I tried adding some data, adding data from my app and even putting the computer into sleep mode and coming back out.

So not sure what it is but it is either time related or caused by something else.

[madmacc]

I just had a similar issue after leaving my computer for a couple of hours in sleep mode. Slightly different error this time and at least it didn't delete the files this time!

Log is attached. Looks the same. I'm not sure if this is correct log file as it doesn't have much interesting in it.

i emulators: Received SIGINT (Ctrl-C) for the first time. Starting a clean shutdown. i emulators: Please wait for a clean shutdown or send the SIGINT (Ctrl-C) signal again to stop right now. i Automatically exporting data using --export-on-exit "emulator/emulator-export-data" please wait for the export to finish... ! Did not find any running emulators for project hivepass-dev. ! Automatic export to "emulator/emulator-export-data" failed, going to exit now... i emulators: Shutting down emulators. i ui: Stopping Emulator UI ! Emulator UI has exited upon receiving signal: SIGINT i hosting: Stopping Hosting Emulator i firestore: Stopping Firestore Emulator ! Firestore Emulator has exited upon receiving signal: SIGINT i auth: Stopping Authentication Emulator i hub: Stopping emulator hub ! hub: Error stopping emulator hub i logging: Stopping Logging Emulator firestore-debug.log

[yuchenshi]

! Did not find any running emulators for project hivepass-dev.

The Emulator Suite writes a temporary file to keep track of the running instances, and looks like that file was deleted somehow before you terminated the suite. For example, Windows may delete those temporary files to reclaim disk space (especially if disk space is low), or it may be some other cleanup tools installed on your dev machine.

Right now, --export-on-exit relies on that file, but it doesn't have to -- I've created b/179048252 to track making it more reliable. Thanks again for the bug report!

[madmacc]

@yuchenshi Ok that 2nd error above is because I am running two instances of the emulator. If I stop the functions emulator first it causes that problem. This is not the problem where it deletes the exported data files, it just fails to export. My cloud functions folder is in a different project directory so I have to run the functions emulator as a separate instance.

Unfortunately it doesn't resolved my original problem of it deleting the files though.

[yuchenshi]

Glad we found that out! The export APIs cannot handle two instances running together yet, but the internal bug I've created above should be able to solve this.

In the meantime, have you considered creating symlinks so that your functions can be "part of" the main project directory? They are available on Windows 10 and you can ignore them in .gitignore (or similar) if you'd like to.

[madmacc]

In the meantime, have you considered creating symlinks so that your functions can be "part of" the main project directory? They are available on Windows 10 and you can ignore them in .gitignore (or similar) if you'd like to.

This is a good idea. I have not used them before but have had them on my todo list, I will give it a try. I tried a shortcut which of course did not work!

It must be common scenario for people that have multiple apps in their firebase project. Not sure why I am the only one who has brought it up.

[madmacc]

Actually after some more testing the first issue with the deleted export data files is due to the two instances of the emulator running. But only if I start the firestore emulator first then the functions emulator and then stop the firestore emulator while the functions emulator is still running.

UPDATE: It seems to work well with symbolic links. I will update if any issues.

If anyone else want to do this the command on Windows is: mklink /D "R:\firestore-project\functions" "R:\cloud-functions-proj\functions"

[JoeEarly]

Am seeing this also on Mac, every other time my export files are either corrupted or missing data.

Will comeback with the detail

[carloschida]

On macOS 11.1 with firebase-tools@9.3.0 I'm seeing something of the kind:

On WebStorm 2020.3.2 when using the "Run" button on emulators:start:

// package.json

    "emulators:start": "firebase emulators:start --project=\"local-emulators\" --import=\"./a\" --export-on-exit=\"./a\""

The result is:

/Users/carlos.chida/.nvm/versions/node/v12.13.0/bin/node /usr/local/Cellar/yarn/1.22.5/libexec/bin/yarn.js run emulators:start
yarn run v1.22.10
$ firebase emulators:start --project="local-emulators" --import="./a" --export-on-exit="./a"
...
┌─────────────────────────────────────────────────────────────┐
│ ✔  All emulators ready! It is now safe to connect your app. │
│ i  View Emulator UI at http://localhost:4000                │
└─────────────────────────────────────────────────────────────┘

### STOPPED THE PROCESS VIA WEBSTORM ###

i  emulators: Received SIGINT (Ctrl-C) for the first time. Starting a clean shutdown.
i  emulators: Please wait for a clean shutdown or send the SIGINT (Ctrl-C) signal again to stop right now.
i  Automatically exporting data using --export-on-exit "./a" please wait for the export to finish...
⚠  Emulator UI has exited upon receiving signal: SIGINT
i  Found running emulator hub for project local-emulators at http://localhost:4400
i  Creating export directory /Users/carlos.chida/WebstormProjects/veriphi/functions/a
i  Exporting data to: /Users/carlos.chida/WebstormProjects/veriphi/functions/a

But only the folder gets created without any contents. I'm assuming that WebStorm is not giving the process enough time after sending the interrupt signal.

On the other hand, when running yarn emulators:start, it works without problems. Nonetheless, it looks like it consumes much time after receiving the signal:

yarn emulators:start
yarn run v1.22.10
$ firebase emulators:start --project="local-emulators" --import="./a" --export-on-exit="./a"
...
### PRESSED ctrl-C 
i  emulators: Received SIGINT (Ctrl-C) for the first time. Starting a clean shutdown.
i  emulators: Please wait for a clean shutdown or send the SIGINT (Ctrl-C) signal again to stop right now.
i  Automatically exporting data using --export-on-exit "./import-export-data" please wait for the export to finish...
i  Found running emulator hub for project local-emulators at http://localhost:4400                                                                                                                                     
i  Creating export directory /Users/carlos.chida/WebstormProjects/veriphi/functions/import-export-data
i  Exporting data to: /Users/carlos.chida/WebstormProjects/veriphi/functions/import-export-data
i  emulators: Received export request. Exporting data to /Users/carlos.chida/WebstormProjects/veriphi/functions/import-export-data.
$ ### NEW PROMPT!
➜ ✔  emulators: Export complete.
✔  Export complete
i  emulators: Shutting down emulators.
i  ui: Stopping Emulator UI
⚠  Emulator UI has exited upon receiving signal: SIGINT
i  functions: Stopping Functions Emulator
i  database: Stopping Database Emulator
i  firestore: Stopping Firestore Emulator
i  pubsub: Stopping Pub/Sub Emulator
⚠  functions: The Cloud Firestore emulator is not running, so calls to Firestore will affect production.
i  auth: Stopping Authentication Emulator
i  hub: Stopping emulator hub
$ ### ANOTHER NEW PROMPT

[yuchenshi]

@carloschida Thanks for the report! Exporting may take a few seconds and if WebStorm isn't giving the emulator enough time to do the work, there isn't a ton we can do here. We suggest running emulators:start in a terminal and you may find the built-in WebStorm terminal emulator better than the "Run" button for your case. Anyway, feel free to contact WebStorm regarding the issue.

[wceolin]

Not sure if it's related to this issue but a similar error is happening to a co-worker:

emulators: Export failed: Server Error. read ECONNRESET

Our emulator script:

firebase --emulators:exec --import="./exports/current" --export-on-exit --ui "next dev"

It's very hard to reproduce since it happens only sometimes. I haven't been able to reproduce it myself but I was wondering if you have any ideas of what this could be or what I could investigate.

Another thing I'm finding weird is that we have duplicated shutting down operations:

emulators: Received export request. Exporting data to /exports/current.
✔  emulators: Export complete.
✔  emulators: Export complete.
✔  Export complete
i  emulators: Shutting down emulators.
i  ui: Stopping Emulator UI
✔  Export complete
i  emulators: Shutting down emulators.
i  ui: Stopping Emulator UI
⚠  Emulator UI has exited upon receiving signal: SIGINT
i  functions: Stopping Functions Emulator
i  functions: Stopping Functions Emulator
i  database: Stopping Database Emulator
i  database: Stopping Database Emulator
i  firestore: Stopping Firestore Emulator
i  firestore: Stopping Firestore Emulator
i  pubsub: Stopping Pub/Sub Emulator
i  pubsub: Stopping Pub/Sub Emulator
i  auth: Stopping Authentication Emulator
i  auth: Stopping Authentication Emulator
i  hub: Stopping emulator hub
i  hub: Stopping emulator hub
i  logging: Stopping Logging Emulator
i  logging: Stopping Logging Emulator

You can see all operation happen twice. However, that seems to be related to the next dev script used by Next.js. That doesn't happen with other scripts (i.e. Jest in watch mode).

[yuchenshi]

@wceolin This is an interesting case and would you be able to get the firebase-debug.log for this one? That may be helpful for troubleshooting what caused everything to happen twice.

[wceolin]

@yuchenshi These are the logs my co-worker shared with me when she got that error:

Apr 27, 2021 4:50:13 PM com.google.cloud.datastore.emulator.firestore.webchannel.FirestoreV1WebChannelAdapter$FirestoreWriteHandler onClose
INFO: channel closed
Apr 27, 2021 4:50:43 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
INFO: Detected non-HTTP/2 connection.
Apr 27, 2021 4:51:43 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
INFO: Detected non-HTTP/2 connection.
Apr 27, 2021 4:52:28 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
INFO: Detected non-HTTP/2 connection.
Apr 27, 2021 5:13:26 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
INFO: Detected HTTP/2 connection.
Apr 27, 2021 5:13:33 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
INFO: Detected non-HTTP/2 connection.
Apr 27, 2021 5:13:34 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
INFO: Detected non-HTTP/2 connection.
*** shutting down gRPC server since JVM is shutting down
*** server shut down

[yuchenshi]

@wceolin I appreciate the help, but we need the firebase-debug.log (created by CLI), not firestore-.

[wceolin]

@yuchenshi Sorry. I wasn't able to find a firebase-debug.log file. There's only database-debug, firestore-debug, pubsub-debug, and ui-debug. Not sure if this helps but the database-debug.log had the following warning:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by io.netty.util.internal.ReflectionUtil (file:/Users/$username/.cache/firebase/emulators/firebase-database-emulator-v4.7.2.jar) to field sun.nio.ch.SelectorImpl.selectedKeys
WARNING: Please consider reporting this to the maintainers of io.netty.util.internal.ReflectionUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
19:10:08.230 [NamespaceSystem-akka.actor.default-dispatcher-5] INFO akka.event.slf4j.Slf4jLogger - Slf4jLogger started
19:10:08.387 [main] INFO com.firebase.server.forge.App$ - Listening at 0.0.0.0:9000
19:35:26.787 [Thread-0] INFO com.firebase.server.forge.App$ - Attempting graceful shutdown.
19:35:26.799 [NamespaceSystem-akka.actor.default-dispatcher-10] INFO com.firebase.core.namespace.Terminator$Terminator - 1 actors left to terminate: project-name
19:35:26.805 [NamespaceSystem-akka.actor.default-dispatcher-10] INFO com.firebase.core.namespace.NamespaceActor - Stopping namespace actor for project-name
19:35:26.806 [NamespaceSystem-akka.actor.default-dispatcher-10] INFO com.firebase.core.namespace.NamespaceActor - Gauges removed for project-name
19:35:26.810 [Thread-0] INFO com.firebase.server.forge.App$ - Graceful shutdown complete.

Btw, this issue doesn't seem to happen when using emulators:start - only when using emulators:exec.

[muellerdberlin]

On macOS 11.1 with firebase-tools@9.3.0 I'm seeing something of the kind:

On WebStorm 2020.3.2 when using the "Run" button on emulators:start:

As mentioned, in Webstorm I could handle it by using the Webstorm terminal and not the Run Configuration. However, that's strange.

[iplaksiy]

This happens intermittently to me as well but only if I leave the emulators running for a few days and the machine goes through various wake and sleep cycles during that time.

[madmacc]

I just came back to a project I had not run in the emulator for a few weeks and the data was gone again. Not sure how or when it happened. I assume it just didn't export correctly when I shut it down the last time. Can be quite frustrating as I need to regularly make a manual backup of the data in case this happens.

[wceolin]

I'm still not able to consistently reproduce the "export failed" error but this issue is happening to 3 co-workers at the moment. In all cases, there's no firebase-debug.log file. Enabling the --debug flag, we got the following information:

emulators: Export failed: Server Error. read ECONNRESET
[2021-07-15T15:00:42.106Z] <<< HTTP RESPONSE 500 {"x-powered-by":"Express","vary":"Origin","content-type":"application/json; charset=utf-8","content-length":"43","etag":"W/\"2b-PTnaVGW4PHglIRlUWNl1/Oxt2Ys\"","date":"Thu, 15 Jul 2021 15:00:42 GMT","connection":"keep-alive","keep-alive":"timeout=5"}
[2021-07-15T15:00:42.107Z] <<< HTTP RESPONSE BODY Server Error. read ECONNRESET {"message":"Server Error. read ECONNRESET"}
⚠  Export request failed, see emulator logs for more information. {"name":"FirebaseError","children":[],"exit":1,"message":"Export request failed, see emulator logs for more information.","original":{"name":"FirebaseError","children":[],"context":{"body":{"message":"Server Error. read ECONNRESET","error":{"message":"Unknown Error"}},"response":{"statusCode":500,"body":{"message":"Server Error. read ECONNRESET","error":{"message":"Unknown Error"}},"headers":{"x-powered-by":"Express","vary":"Origin","content-type":"application/json; charset=utf-8","content-length":"43","etag":"W/\"2b-PTnaVGW4PHglIRlUWNl1/Oxt2Ys\"","date":"Thu, 15 Jul 2021 15:00:42 GMT","connection":"keep-alive","keep-alive":"timeout=5"},"request":{"uri":{"protocol":"http:","slashes":true,"auth":null,"host":"localhost:4400","port":"4400","hostname":"localhost","hash":null,"search":null,"query":null,"pathname":"/_admin/export","path":"/_admin/export","href":"http://localhost:4400/_admin/export"},"method":"POST"}}},"exit":2,"message":"HTTP Error: 500, Unknown Error","status":500},"status":500}
⚠  Automatic export to "./firebase-sdk/exports/current" failed, going to exit now...

Sometimes, we also get this:

logging: Stopping Logging Emulator
(node:3153) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'exit' of undefined
    at /my-project/node_modules/firebase-tools/lib/command.js:92:40
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
(node:3153) 

That seems to come from this line suggesting that maybe the error is undefined?

These export failed issues seem to happening only when using the emulators:exec. We haven't seen them with emulators:start AFAIK.

I'm going to try digging more into the firebase-tools codebase to see if I can have a better understanding of what might be going on here but I'm posting the information above in case someone more familiar with the codebase has any ideas.

[scotteau]

Can confirm the problem occurs when running firebase emulator within webstorm run window, but works fine in terminal.

[wceolin]

I bought a new MacBook Air this week and the export failed error is happening every time on my new machine (I'm using the terminal, btw), which is weird because it's basically the same setup I have in another machine (a Mac Mini M1) where it doesn't happen.

Initially, I thought this could be because I was using Node 14 on my new machine and Node 12 in my old one (that's the only difference I could spot) but that's not the case since a co-worker told me she's having the same issue using Node 12.

This also doesn't seem to be OS specific. This is happening to two co-workers using Ubuntu, to another one using MacOS with an Intel processor, and now with me using MacOS with an M1 processor.

[50an6xy06r6n]

@yuchenshi is there a way to recover from this state? I seem to run into it occasionally when I make changes to my code that crash the function (I'm running tsc in watch mode).

[MorenoMdz]

We have this issue as well when we leave the emulator running for long times, is there a way to recover from this? The file the emulator writes can be saved somehow to be reused later when this happen in case it gets removed?

[Drew-Daniels]

I have been having the same issue on Windows 10 (Firebase Tools version 10.7.1). It happens regardless of if the emulator has been running for a long time or just a minute or two, and when the amount of data is minimal (i.e., one collection with one document with one field). I try exporting emulator data to a local folder directory named data but when I shut down I get the following exported to my root directory:

When I run firebase emulators:start --export-on-exit=data, this is my PowerShell output:

i  emulators: Starting emulators: auth, firestore, database, hosting, pubsub, storage
!  emulators: It seems that you are running multiple instances of the emulator suite for project drew-daniels-wheres-waldo. This may result in unexpected behavior.
i  firestore: Firestore Emulator logging to firestore-debug.log
i  database: Database Emulator logging to database-debug.log
i  pubsub: Pub/Sub Emulator logging to pubsub-debug.log
i  hosting: Serving hosting files from: build
+  hosting: Local server: http://localhost:5000
i  ui: Emulator UI logging to ui-debug.log

┌─────────────────────────────────────────────────────────────┐
│ ✔  All emulators ready! It is now safe to connect your app. │
│ i  View Emulator UI at http://localhost:4000                │
└─────────────────────────────────────────────────────────────┘

┌────────────────┬────────────────┬─────────────────────────────────┐
│ Emulator       │ Host:Port      │ View in Emulator UI             │
├────────────────┼────────────────┼─────────────────────────────────┤
│ Authentication │ localhost:9099 │ http://localhost:4000/auth      │
├────────────────┼────────────────┼─────────────────────────────────┤
│ Firestore      │ localhost:8080 │ http://localhost:4000/firestore │
├────────────────┼────────────────┼─────────────────────────────────┤
│ Database       │ localhost:9000 │ http://localhost:4000/database  │
├────────────────┼────────────────┼─────────────────────────────────┤
│ Hosting        │ localhost:5000 │ n/a                             │
├────────────────┼────────────────┼─────────────────────────────────┤
│ Pub/Sub        │ localhost:8085 │ n/a                             │
├────────────────┼────────────────┼─────────────────────────────────┤
│ Storage        │ localhost:9199 │ http://localhost:4000/storage   │
└────────────────┴────────────────┴─────────────────────────────────┘
  Emulator Hub running at localhost:4400
  Other reserved ports: 4500

Issues? Report them at https://github.com/firebase/firebase-tools/issues and attach the *-debug.log files.

Everything appears to run fine, but once I ctrl+C it appears that the emulator does not shut down gracefully:

i  emulators: Received SIGINT (Ctrl-C) for the first time. Starting a clean shutdown.
i  emulators: Please wait for a clean shutdown or send the SIGINT (Ctrl-C) signal again to stop right now.
i  Automatically exporting data using --export-on-exit "data" please wait for the export to finish...

Terminate batch job (Y/N)? Error: Storage Emulator Rules runtime exited unexpectedly.
i  Found running emulator hub for project (my-project-name) at http://localhost:4400
i  Exporting data to: C:\Users\MyUserName\my\folder\project\data
i  emulators: Received export request. Exporting data to C:\Users\MyUserName\my\folder\project\data.

What I have tried:

• Changing the relative file paths from ./data to data
• Ensuring all java.exe instances are closed (via task manager)
• Restarting my computer

Made an SO post on this yesterday: https://stackoverflow.com/questions/71972842/firebase-emulator-export-on-exit-not-working-properly-windows-10?noredirect=1#comment127180514_71972842

[nayfin]

My co-worker was having the exact same issue as described above by @Drew-Daniels , and I wasn't. He was also seeing 2 java terminal's open, but I wasn't. After upgrading to the latest firebase-tools I started getting a warning that I need to upgrade Java to at least v11 (not sure what I had initially or why it was so old). So I upgrade Java to v18, now I am seeing the 2 java terminals open when the emulator starts, and my emulator data is getting exported to a folder at my project root instead of the export location. Before upgrading Java it seemed to create this root folder for a few seconds on shutdown, then move the contents over, now it seems like something is interrupting the process halfway through. I'm tagging @yuchenshi and @weixifan since this seems to at give some insight into a change that can initiate the issue. Let me know if I should create a new issue or if you think this is related.

Update: I tried rolling back Java JDK to v11, but that didn't help. Looks like the Java terminal's don't like how they are being shut down, which is maybe what is causing the process of moving the export files to the desired folder not to happen.

Here is what the Java terminals look like before shutting down.

And here is after hitting ctrl + c from the terminal running the emulator.

[abakucz]

I do experience this issue as well.

And it also started after I updated Java to 18 (and also upgraded NodeJS 14 from 12).

[nayfin]

I just noticed that @Drew-Daniels was getting the ! emulators: It seems that you are running multiple instances of the emulator suite... error. Which seems like it's common among people having this issue of the data not being exported where expected. I am not getting error on initialization, but I'm still having the data export issue.

No error on initialization

But data is still being created in hashed folder instead of in the specified folder:

[abderahmaneBasil]

Same problem here but with a different context. I dockerized the emulators and wanted to export the data in a volume when container exits but it fails every time with the following error :

[dereekb]

On the Docker issue, I ran into the same thing, but it unable to save was a “problem” with Docker rather than the emulator.

Your Firebase data folder can’t be the mapped folder, you need to map a parent folder and try and export to a child folder.

Here’s my mappings and usage:

https://github.com/dereekb/dbx-components/blob/develop/docker-compose.yml

https://github.com/dereekb/dbx-components/blob/7a5e05a27ab83976ba7dec6430477c8ecd458416/workspace.json#L685

Still, I’ll occasionally have the emulator dump an empty folder in the root folder, and I’m not sure exactly what causes that.

[abderahmaneBasil]

Thanks @dereekb 🙌 It was as you wrote : I mapped to a parent folder and it works perfectly now.

[nayfin]

Can someone with a working instance please lookup and post your versions for the following dependencies: Java, node, firebase, firebase-tools? Mapping to a parent folder didn't work for me and we're not using docker. This is the emulator start command we run: firebase emulators:start --import=./emulator/export-data --export-on-exit=./emulator/export-data

Another note, exporting manually while running works fine using: firebase emulators:export ./emulator/export-data

But it does get interrupted if there is already data in the export folder and throws a prompt:

The directory C:\some-directory already contains export data. Exporting again to the same directory will overwrite all data. Do you want to continue?

Maybe this command is being used internally and the prompt is screwing something up? I've manually exported before and I don't remember the prompt being there in the past.

[ntorrey]

Same problem and behavior as @nayfin Windows 11 java 17.0.1 node 16.13.1 firebase 9.8.1 firebase-tools 11.0.1

[Darkade]

I'm guessing this is related but running firebase emulators:export . just deleted my whole repo after failing to run. I understand if this isn't the syntax the command expected but deleting the whole repo?!

[yuchenshi]

@Darkade We've tried to make the command not do that. Please make sure you're using the latest Firebase CLI version. If that still happens, please see https://github.com/firebase/firebase-tools/issues/3962#issuecomment-1128169529 for a longer discussion and feel free to provide any relevant information there.

Friendly reminder to keep this discussion on export failures. If you have any information to add on this front, even just OS / Java / Node versions, feel free to add a comment to help us pinpoint this. (Right now, it looks related to some Java 17/18 changes on specific platforms -- more info appreciated.) If you have a different problem, don't hesitate to open a new issue and we'll help you there.

[Darkade]

I mean, the export did failed before what happened @yuchenshi, but I get you, thanks. If it helps these are the versions I'm using

$ java --version
openjdk 11.0.15 2022-04-19
OpenJDK Runtime Environment Homebrew (build 11.0.15+0)
OpenJDK 64-Bit Server VM Homebrew (build 11.0.15+0, mixed mode)
$ node --version
v14.19.3
$ firebase --version
11.0.1
$  arch
arm64
$ sw_vers
ProductName:    macOS
ProductVersion: 12.4
BuildVersion:   21F79

[abakucz]

@yuchenshi

Friendly reminder to keep this discussion on export failures. If you have any information to add on this front, even just OS / Java / Node versions, feel free to add a comment to help us pinpoint this. (Right now, it looks related to some Java 17/18 changes on specific platforms -- more info appreciated.) If you have a different problem, don't hesitate to open a new issue and we'll help you there.

That seems to be problem in my case. This failures started to happen when I updated Node and Java.

λ node --version
v14.19.1

λ java --version
java 18 2022-03-22
Java(TM) SE Runtime Environment (build 18+36-2087)
Java HotSpot(TM) 64-Bit Server VM (build 18+36-2087, mixed mode, sharing)

λ systeminfo
OS Name:                   Microsoft Windows 10 Home
OS Version:                10.0.19043 N/A Build 19043

[rhuanbarros]

I have the same issue.

`[debug] [2022-06-04T19:12:25.000Z] ---------------------------------------------------------------------- [debug] [2022-06-04T19:12:25.002Z] Command: C:\Program Files\nodejs\node.exe C:\Program Files\nodejs\node_modules\firebase-tools\lib\bin\firebase.js emulators:start --import=./exported-data --export-on-exit [debug] [2022-06-04T19:12:25.002Z] CLI Version: 11.0.1 [debug] [2022-06-04T19:12:25.002Z] Platform: win32 [debug] [2022-06-04T19:12:25.003Z] Node Version: v16.15.1 [debug] [2022-06-04T19:12:25.003Z] Time: Sat Jun 04 2022 16:12:25 GMT-0300 (Horário Padrão de Brasília) [debug] [2022-06-04T19:12:25.004Z] ---------------------------------------------------------------------- [debug] [debug] [2022-06-04T19:12:25.065Z] > command requires scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","https://www.googleapis.com/auth/cloud-platform"] [debug] [2022-06-04T19:12:25.066Z] > authorizing via signed-in user (materialeshopdev@gmail.com) [debug] [2022-06-04T19:12:25.192Z] java version "18.0.1.1" 2022-04-22 Java(TM) SE Runtime Environment (build 18.0.1.1+2-6)

[debug] [2022-06-04T19:12:25.193Z] Java HotSpot(TM) 64-Bit Server VM (build 18.0.1.1+2-6, mixed mode, sharing)

[debug] [2022-06-04T19:12:25.212Z] Parsed Java major version: 18 [info] i emulators: Starting emulators: auth, functions, firestore {"metadata":{"emulator":{"name":"hub"},"message":"Starting emulators: auth, functions, firestore"}} [debug] [2022-06-04T19:12:25.242Z] [hub] writing locator at C:\Users\rhuan\AppData\Local\Temp\hub-materialeshop-dev.json [warn] ! functions: The following emulators are not running, calls to these services from the Functions emulator will affect production: database, hosting, pubsub, storage {"metadata":{"emulator":{"name":"functions"},"message":"The following emulators are not running, calls to these services from the Functions emulator will affect production: \u001b[1mdatabase, hosting, pubsub, storage\u001b[22m"}} [info] + functions: Using node@16 from host. {"metadata":{"emulator":{"name":"functions"},"message":"Using node@16 from host."}} [debug] [2022-06-04T19:12:25.284Z] defaultcredentials: writing to file C:\Users\rhuan\AppData\Roaming\firebase\materialeshopdev_gmail_com_application_default_credentials.json [debug] [2022-06-04T19:12:25.286Z] Setting GAC to C:\Users\rhuan\AppData\Roaming\firebase\materialeshopdev_gmail_com_application_default_credentials.json {"metadata":{"emulator":{"name":"functions"},"message":"Setting GAC to C:\Users\rhuan\AppData\Roaming\firebase\materialeshopdev_gmail_com_application_default_credentials.json"}} [debug] [2022-06-04T19:12:25.289Z] >>> [apiv2][query] GET https://firebase.googleapis.com/v1beta1/projects/materialeshop-dev/adminSdkConfig [none] [debug] [2022-06-04T19:12:26.422Z] <<< [apiv2][status] GET https://firebase.googleapis.com/v1beta1/projects/materialeshop-dev/adminSdkConfig 200 [debug] [2022-06-04T19:12:26.422Z] <<< [apiv2][body] GET https://firebase.googleapis.com/v1beta1/projects/materialeshop-dev/adminSdkConfig {"projectId":"materialeshop-dev","storageBucket":"materialeshop-dev.appspot.com","locationId":"southamerica-east1"} [debug] [2022-06-04T19:12:26.441Z] Ignoring unsupported arg: projectId {"metadata":{"emulator":{"name":"firestore"},"message":"Ignoring unsupported arg: projectId"}} [debug] [2022-06-04T19:12:26.441Z] Ignoring unsupported arg: auto_download {"metadata":{"emulator":{"name":"firestore"},"message":"Ignoring unsupported arg: auto_download"}} [debug] [2022-06-04T19:12:26.442Z] Starting Firestore Emulator with command {"binary":"java","args":["-Dgoogle.cloud_firestore.debug_log_level=FINE","-Duser.language=en","-jar","C:\Users\rhuan\.cache\firebase\emulators\cloud-firestore-emulator-v1.14.3.jar","--host","0.0.0.0","--port",8080,"--rules","D:\MaterialeCode\materialeshop-backend-2\firestore.rules","--functions_emulator","0.0.0.0:5001"],"optionalArgs":["port","webchannel_port","host","rules","functions_emulator","seed_from_export"],"joinArgs":false} {"metadata":{"emulator":{"name":"firestore"},"message":"Starting Firestore Emulator with command {\"binary\":\"java\",\"args\":[\"-Dgoogle.cloud_firestore.debug_log_level=FINE\",\"-Duser.language=en\",\"-jar\",\"C:\\Users\\rhuan\\.cache\\firebase\\emulators\\cloud-firestore-emulator-v1.14.3.jar\",\"--host\",\"0.0.0.0\",\"--port\",8080,\"--rules\",\"D:\\MaterialeCode\\materialeshop-backend-2\\firestore.rules\",\"--functions_emulator\",\"0.0.0.0:5001\"],\"optionalArgs\":[\"port\",\"webchannel_port\",\"host\",\"rules\",\"functions_emulator\",\"seed_from_export\"],\"joinArgs\":false}"}} [info] i firestore: Firestore Emulator logging to firestore-debug.log {"metadata":{"emulator":{"name":"firestore"},"message":"Firestore Emulator logging to \u001b[1mfirestore-debug.log\u001b[22m"}} [debug] [2022-06-04T19:12:42.178Z] Ignoring unsupported arg: auto_download {"metadata":{"emulator":{"name":"ui"},"message":"Ignoring unsupported arg: auto_download"}} [debug] [2022-06-04T19:12:42.178Z] Ignoring unsupported arg: port {"metadata":{"emulator":{"name":"ui"},"message":"Ignoring unsupported arg: port"}} [debug] [2022-06-04T19:12:42.179Z] Starting Emulator UI with command {"binary":"node","args":["--dns-result-order=ipv4first","C:\Users\rhuan\.cache\firebase\emulators\ui-v1.7.0\server.bundle.js"],"optionalArgs":[],"joinArgs":false} {"metadata":{"emulator":{"name":"ui"},"message":"Starting Emulator UI with command {\"binary\":\"node\",\"args\":[\"--dns-result-order=ipv4first\",\"C:\\Users\\rhuan\\.cache\\firebase\\emulators\\ui-v1.7.0\\server.bundle.js\"],\"optionalArgs\":[],\"joinArgs\":false}"}} [info] i ui: Emulator UI logging to ui-debug.log {"metadata":{"emulator":{"name":"ui"},"message":"Emulator UI logging to \u001b[1mui-debug.log\u001b[22m"}} [debug] [2022-06-04T19:12:42.437Z] Web / API server started at localhost:4000 {"metadata":{"emulator":{"name":"ui"},"message":"Web / API server started at localhost:4000\n"}} [info] i functions: Watching "D:\MaterialeCode\materialeshop-backend-2\functions" for Cloud Functions... {"metadata":{"emulator":{"name":"functions"},"message":"Watching \"D:\MaterialeCode\materialeshop-backend-2\functions\" for Cloud Functions..."}} [debug] [2022-06-04T19:12:42.471Z] Validating nodejs source [debug] [2022-06-04T19:12:46.476Z] > [functions] package.json contents: { "name": "functions", "scripts": { "lint": "eslint --ext .js,.ts .", "build": "tsc", "serve": "npm run build && firebase emulators:start --only functions", "shell": "npm run build && firebase functions:shell", "start": "npm run shell", "deploy": "firebase deploy --only functions", "logs": "firebase functions:log", "s": "firebase emulators:start --import=./saved-data --export-on-exit", "s2": "firebase emulators:start --import=./exported-data --export-on-exit", "bw": "tsc -w", "bws": "npm run bw | firebase emulators:start --import=./saved-data --export-on-exit", "export": "firebase emulators:export ./exported-data" }, "engines": { "node": "16" }, "main": "lib/index.js", "dependencies": { "firebase-admin": "^10.0.2", "firebase-functions": "^3.18.0" }, "devDependencies": { "@typescript-eslint/eslint-plugin": "^5.12.0", "@typescript-eslint/parser": "^5.12.0", "eslint": "^8.9.0", "eslint-config-google": "^0.14.0", "eslint-plugin-import": "^2.25.4", "firebase-functions-test": "^0.2.0", "typescript": "^4.5.4" }, "private": true } [debug] [2022-06-04T19:12:46.477Z] Building nodejs source [debug] [2022-06-04T19:12:46.477Z] Analyzing nodejs backend spec [debug] [2022-06-04T19:12:46.478Z] Could not find functions.yaml. Must use http discovery [debug] [2022-06-04T19:12:46.892Z] Serving at port 9005

[debug] [2022-06-04T19:12:47.042Z] Got response from //functions.yaml {"endpoints":{"selecionarLojas":{"platform":"gcfv1","httpsTrigger":{},"entryPoint":"selecionarLojas"},"criaPedidoOrcamento":{"platform":"gcfv1","httpsTrigger":{},"entryPoint":"criaPedidoOrcamento"}},"specVersion":"v1alpha1","requiredAPIs":[]} [debug] [2022-06-04T19:12:47.051Z] shutdown requested via //quitquitquit

[info] + functions[us-central1-selecionarLojas]: http function initialized (http://0.0.0.0:5001/materialeshop-dev/us-central1/selecionarLojas). {"metadata":{"emulator":{"name":"functions"},"message":"\u001b[1mhttp\u001b[22m function initialized (http://0.0.0.0:5001/materialeshop-dev/us-central1/selecionarLojas)."}} [info] + functions[us-central1-criaPedidoOrcamento]: http function initialized (http://0.0.0.0:5001/materialeshop-dev/us-central1/criaPedidoOrcamento). {"metadata":{"emulator":{"name":"functions"},"message":"\u001b[1mhttp\u001b[22m function initialized (http://0.0.0.0:5001/materialeshop-dev/us-central1/criaPedidoOrcamento)."}} [info] ┌─────────────────────────────────────────────────────────────┐ │ ✔ All emulators ready! It is now safe to connect your app. │ │ i View Emulator UI at http://localhost:4000 │ └─────────────────────────────────────────────────────────────┘

┌────────────────┬──────────────┬─────────────────────────────────┐ │ Emulator │ Host:Port │ View in Emulator UI │ ├────────────────┼──────────────┼─────────────────────────────────┤ │ Authentication │ 0.0.0.0:9099 │ http://localhost:4000/auth │ ├────────────────┼──────────────┼─────────────────────────────────┤ │ Functions │ 0.0.0.0:5001 │ http://localhost:4000/functions │ ├────────────────┼──────────────┼─────────────────────────────────┤ │ Firestore │ 0.0.0.0:8080 │ http://localhost:4000/firestore │ └────────────────┴──────────────┴─────────────────────────────────┘ Emulator Hub running at localhost:4400 Other reserved ports: 4500

Issues? Report them at https://github.com/firebase/firebase-tools/issues and attach the *-debug.log files.

[debug] [2022-06-04T19:14:02.666Z] >>> [apiv2][query] POST http://0.0.0.0:5001/functions/projects/materialeshop-dev/trigger_multicast [none] [debug] [2022-06-04T19:14:02.667Z] >>> [apiv2][body] POST http://0.0.0.0:5001/functions/projects/materialeshop-dev/trigger_multicast {"eventId":"cf269786-d438-4c11-be30-de8ab886ea41","eventType":"providers/firebase.auth/eventTypes/user.create","resource":{"name":"projects/materialeshop-dev","service":"firebaseauth.googleapis.com"},"params":{},"timestamp":"2022-06-04T19:14:02.665Z","data":{"uid":"lxJcOEhWZkbv0Favmb3Qo9BmckeG","email":"usuario1@gmail.com","emailVerified":false,"metadata":{"creationTime":"2022-06-04T19:14:02.664Z","lastSignInTime":"2022-06-04T19:14:02.663Z"},"customClaims":{},"providerData":[{"providerId":"password","email":"usuario1@gmail.com","federatedId":"usuario1@gmail.com","rawId":"usuario1@gmail.com"}]}} [debug] [2022-06-04T19:14:02.677Z] <<< [apiv2][status] POST http://0.0.0.0:5001/functions/projects/materialeshop-dev/trigger_multicast 200 [debug] [2022-06-04T19:14:02.677Z] <<< [apiv2][body] POST http://0.0.0.0:5001/functions/projects/materialeshop-dev/trigger_multicast {"status":"multicast_acknowledged"} [debug] [2022-06-04T19:14:15.686Z] ---------------------------------------------------------------------- [debug] [2022-06-04T19:14:15.688Z] Command: C:\Program Files\nodejs\node.exe C:\Program Files\nodejs\node_modules\firebase-tools\lib\bin\firebase.js emulators:export ./exported-data [debug] [2022-06-04T19:14:15.689Z] CLI Version: 11.0.1 [debug] [2022-06-04T19:14:15.689Z] Platform: win32 [debug] [2022-06-04T19:14:15.689Z] Node Version: v16.15.1 [debug] [2022-06-04T19:14:15.690Z] Time: Sat Jun 04 2022 16:14:15 GMT-0300 (Horário Padrão de Brasília) [debug] [2022-06-04T19:14:15.691Z] ---------------------------------------------------------------------- [debug] [debug] [2022-06-04T19:14:15.770Z] >>> [apiv2][query] GET http://localhost:4400/ [none] [debug] [2022-06-04T19:14:15.790Z] <<< [apiv2][status] GET http://localhost:4400/ 200 [debug] [2022-06-04T19:14:15.791Z] <<< [apiv2][body] GET http://localhost:4400/ {"version":"11.0.1","host":"localhost","port":4400} [info] i Found running emulator hub for project materialeshop-dev at http://localhost:4400 [info] i Creating export directory D:\MaterialeCode\materialeshop-backend-2\functions\exported-data [info] i Exporting data to: D:\MaterialeCode\materialeshop-backend-2\functions\exported-data [debug] [2022-06-04T19:14:15.796Z] >>> [apiv2][query] POST http://localhost:4400/_admin/export [none] [debug] [2022-06-04T19:14:15.796Z] >>> [apiv2][body] POST http://localhost:4400/_admin/export {"path":"D:\MaterialeCode\materialeshop-backend-2\functions\exported-data"} [info] i emulators: Received export request. Exporting data to D:\MaterialeCode\materialeshop-backend-2\functions\exported-data. [debug] [2022-06-04T19:14:15.806Z] >>> [apiv2][query] POST http://0.0.0.0:8080/emulator/v1/projects/materialeshop-dev:export [none] [debug] [2022-06-04T19:14:15.806Z] >>> [apiv2][body] POST http://0.0.0.0:8080/emulator/v1/projects/materialeshop-dev:export {"database":"projects/materialeshop-dev/databases/(default)","export_directory":"firebase-export-1654370055804maz9GA","export_name":"firestore_export"} [debug] [2022-06-04T19:14:15.870Z] File D:\MaterialeCode\materialeshop-backend-2\functions\firebase-export-1654370055804maz9GA\firestore_export\all_namespaces\all_kinds\output-0 changed, reloading triggers {"metadata":{"emulator":{"name":"functions"},"message":"File D:\MaterialeCode\materialeshop-backend-2\functions\firebase-export-1654370055804maz9GA\firestore_export\all_namespaces\all_kinds\output-0 changed, reloading triggers"}} [debug] [2022-06-04T19:14:15.877Z] <<< [apiv2][status] POST http://0.0.0.0:8080/emulator/v1/projects/materialeshop-dev:export 200 [debug] [2022-06-04T19:14:15.877Z] <<< [apiv2][body] POST http://0.0.0.0:8080/emulator/v1/projects/materialeshop-dev:export {} [debug] [2022-06-04T19:14:15.878Z] Exporting auth users in Project materialeshop-dev to firebase-export-1654370055804maz9GA\auth_export\accounts.json [debug] [2022-06-04T19:14:15.892Z] Exporting project config in Project materialeshop-dev to firebase-export-1654370055804maz9GA\auth_export\accounts.json [debug] [2022-06-04T19:14:15.902Z] hubExport: swapping firebase-export-1654370055804maz9GA with D:\MaterialeCode\materialeshop-backend-2\functions\exported-data [warn] ! emulators: Export failed: EPERM: operation not permitted, rename 'firebase-export-1654370055804maz9GA' -> 'D:\MaterialeCode\materialeshop-backend-2\functions\exported-data' [debug] [2022-06-04T19:14:15.909Z] <<< [apiv2][status] POST http://localhost:4400/_admin/export 500 [debug] [2022-06-04T19:14:15.910Z] <<< [apiv2][body] POST http://localhost:4400/_admin/export {"message":"EPERM: operation not permitted, rename 'firebase-export-1654370055804maz9GA' -> 'D:\MaterialeCode\materialeshop-backend-2\functions\exported-data'"} `

[rhuanbarros]

I just recreated the project in WSL Ubuntu, installed java 11, and everything worked just fine.

[nesimtunc]

Experiencing the same issue. None of those solution works for me.

[kiptoomm]

Same problems here. My setup:

$ java -version java version "18.0.1.1" 2022-04-22 Java(TM) SE Runtime Environment (build 18.0.1.1+2-6) Java HotSpot(TM) 64-Bit Server VM (build 18.0.1.1+2-6, mixed mode, sharing)

$ firebase -V 11.1.0

$ sw_vers ProductName: Mac OS X ProductVersion: 10.15.5 BuildVersion: 19F101

$ node --version v16.4.2

[leonelcbraz]

Same here...

[CodyTolene]

Same here.

Node v16.14.2 Firebase v11.2.1

Script ran is firebase emulators:start --only functions,firestore:indexes,firestore --import=./.data --export-on-exit in the terminal and then initiating a clean shutdown with CTRL + C.

Resulting error:

!  emulators: Export failed: EPERM: operation not permitted, rename 'firebase-export-1657573896802aUSBs6' -> 'C:\TestFunctions\.data'
! Export request failed, see emulator logs for more information.

There is nothing logged regarding this issue where stated within "firestore-debug.log". Also the data successfully exports but it's just not to the "./.data/" as in the script but to the path "./firebase-export-1657573896802aUSBs6/" instead.

Please advise. Thank you

[yuchenshi]

Thanks for everyone's input. Looks like we've seen a new trend of permission errors when trying to rename the temporary directory to the final destination. I'm not 100% sure about the permission model of Windows and Linux here, but could it be because that another process (maybe a Java process that failed to exit) still has data files open in the directory? One hypothesis may be that newer Java versions take longer to close files / does not close file handles cleanly on exit.

[dereekb]

I can reproduce the behavior in my dbx-components project. I start the Docker container using ./serve-server.sh, which begins my server instance. It will load content from ./.firebase/data/emulators (mapped to /root/data/emulators in the container) where it is set to import from.

For some extra background, I'm running two functions in parallel, one uses Nx's build/watch feature to rebuild the project when a change occurs, and I've got entr set up to watch for changes to the dist/apps/demo-api folder where Nx dumps the new build, then runs firebase --project=default emulators:start --import=/root/data/emulators --export-on-exit.

Whenever entr sees the dist folder change change, then it sends a shutdown signal (a single SIGTERM) to the emulators.

Here's the logs output for that:

┌─────────────────────────────────────────────────────────────┐
│ ✔  All emulators ready! It is now safe to connect your app. │
│ i  View Emulator UI at http://0.0.0.0:9900                  │
└─────────────────────────────────────────────────────────────┘

┌────────────────┬──────────────┬───────────────────────────────┐
│ Emulator       │ Host:Port    │ View in Emulator UI           │
├────────────────┼──────────────┼───────────────────────────────┤
│ Authentication │ 0.0.0.0:9903 │ http://0.0.0.0:9900/auth      │
├────────────────┼──────────────┼───────────────────────────────┤
│ Functions      │ 0.0.0.0:9902 │ http://0.0.0.0:9900/functions │
├────────────────┼──────────────┼───────────────────────────────┤
│ Firestore      │ 0.0.0.0:9904 │ http://0.0.0.0:9900/firestore │
├────────────────┼──────────────┼───────────────────────────────┤
│ Hosting        │ 0.0.0.0:9901 │ n/a                           │
├────────────────┼──────────────┼───────────────────────────────┤
│ Pub/Sub        │ 0.0.0.0:9905 │ n/a                           │
├────────────────┼──────────────┼───────────────────────────────┤
│ Storage        │ 0.0.0.0:9906 │ http://0.0.0.0:9900/storage   │
└────────────────┴──────────────┴───────────────────────────────┘
  Emulator Hub running at localhost:4400
  Other reserved ports: 4500

Issues? Report them at https://github.com/firebase/firebase-tools/issues and attach the *-debug.log files.

chunk (runtime: main) main.js (main) 704 KiB [entry] [rendered] // <----- Nx finished another build
webpack compiled successfully (045a918f5004ecc7)

i  emulators: Received SIGTERM for the first time. Starting a clean shutdown.  // <--- entr sends shutdown to emulators
i  emulators: Please wait for a clean shutdown or send the SIGTERM signal again to stop right now.
i  Automatically exporting data using --export-on-exit "/root/data/emulators" please wait for the export to finish...
i  Found running emulator hub for project dereekb-components at http://localhost:4400
i  Exporting data to: /root/data/emulators
i  emulators: Received export request. Exporting data to /root/data/emulators.

Waiting for ports 9900-9906 to be available.                        // <--- wait-for-ports.sh run to wait for emulators to shut down
Ports still in use. Waiting for ports to close... PIDs: 1422
1450
1477
1526

Error: Storage Emulator Rules runtime exited unexpectedly.   // <--- weird crash that I'm assuming causes the rest of the emulators to freeze up

Ports still in use. Waiting for ports to close... PIDs: 1450
1477
1526
Ports still in use. Waiting for ports to close... PIDs: 1450
1477
1526
Ports still in use. Waiting for ports to close... PIDs: 1450
1477
1526
Ports still in use. Waiting for ports to close... PIDs: 1450
1477
1526
Preparing to forcefully kill ports...  // <--- wait-for-ports.sh kills the remaining 3 processes

It seems like rules emulator fails to close properly, then causes the others to seize up and not finish their own exports and close until my wait-for-ports.sh script forcefully closes them.

I entered the container on Docker and listed the processes that it is waiting on:

The ones stuck open correspond to Pub/Sub (Port 9905), Firestore (9904) and the UI (9900).

Here's a list of the failed exports that have accumulated as an example of the structure being saved:

A new folder is created each time I make a backend change that Nx captures.

Here's the logs for this test:

logs.txt

[abakucz]

One more possible hint to track this down. I have two projects on local machine (i.e. same NodeJS and Java versions) and one behaves correctly while the other one exhibits the problem discussed here.

The clue might lie in firebase libraries versions:

Project A (export on exit SUCCEEDS):

    "firebase-admin": "^10.1.0",
    "firebase-functions": "^3.20.1",

Project B (export on exit FAILS)

    "firebase-admin": "^10.2.0",
    "firebase-functions": "^3.21.0",