search

willhlaw / node-firestore-backup-restore

Google Firebase Firestore backup and restore tool
91 stars 24 forks source link

Socket hang up error during restore #21

Open NaturalLam opened 6 years ago

NaturalLam commented 6 years ago

I am restoring a DB with about 100K records and 77MB, I will hit this error soon after the restore starts.

!!! Uh-Oh, error saving collection xxx.json { Error: Getting metadata from plugin failed with error: socket hang up at /abc/server/firebase/scripts/node_modules/grpc/src/client.js:554:15 code: 16, metadata: Metadata { _internal_repr: {} } }

Looks like it's the same issue as described here, the write request exceed the Firestore's quota and the server is rejecting most of them. Anyone else hit this issue?

I hit this for backup as well, but not as frequent.

HenrySeed commented 6 years ago

Just hit the same issue with both a 13MB DB and a 58MD DB

willhlaw commented 6 years ago

This PR#20 that I'm reviewing may help as it chunks the files read from the filesystem and it looks like it will have the side effect of chunking the writes to Firestore, basically what the SO link that @NaturalLam posted arrived at.

Other than that, I'm in favor of waiting for the Firebase Firestore team to implement some auto-batching to Firestore as @samtstern mentions here or looking at another PR that solves this issue.