Bandwidth Exhausted

[ronykrell]

Expected behavior

firestore-import working correctly - i.e. importing contents of json file created by firestore-export back into firestore.

Actual behavior

The import works partially - a few seconds after starting I get these errors

Writing documents for dev/CPUqFGRmPqsctsCovM7n/keys
Writing documents for dev/CPUqFGRmPqsctsCovM7n/programCounters
Writing documents for dev/CPUqFGRmPqsctsCovM7n/sheets
Writing documents for dev/CPUqFGRmPqsctsCovM7n/varieties
Error: 8 RESOURCE_EXHAUSTED: Bandwidth exhausted
    at Object.callErrorFromStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call.js:30:26)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client.js:175:52)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:341:141)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:304:181)
    at Http2CallStream.outputStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:116:74)
    at Http2CallStream.maybeOutputStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:155:22)
    at Http2CallStream.endCall (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:141:18)
    at ClientHttp2Stream.<anonymous> (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:403:22)
    at ClientHttp2Stream.emit (events.js:219:5)
    at emitErrorCloseNT (internal/streams/destroy.js:76:8) {
  code: 8,
  details: 'Bandwidth exhausted',
  metadata: Metadata { internalRepr: Map {}, options: {} }
}
Error: 13 INTERNAL: Received RST_STREAM with code 2
    at Object.callErrorFromStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call.js:30:26)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client.js:175:52)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:341:141)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:304:181)
    at Http2CallStream.outputStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:116:74)
    at Http2CallStream.maybeOutputStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:155:22)
    at Http2CallStream.endCall (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:141:18)
    at ClientHttp2Stream.<anonymous> (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:403:22)
    at ClientHttp2Stream.emit (events.js:219:5)
    at emitErrorCloseNT (internal/streams/destroy.js:76:8) {
  code: 13,
  details: 'Received RST_STREAM with code 2',
  metadata: Metadata { internalRepr: Map {}, options: {} }
}
Error: 13 INTERNAL: Received RST_STREAM with code 2
    at Object.callErrorFromStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call.js:30:26)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client.js:175:52)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:341:141)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:304:181)
    at Http2CallStream.outputStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:116:74)
    at Http2CallStream.maybeOutputStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:155:22)
    at Http2CallStream.endCall (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:141:18)
    at ClientHttp2Stream.<anonymous> (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:403:22)
    at ClientHttp2Stream.emit (events.js:219:5)
    at emitErrorCloseNT (internal/streams/destroy.js:76:8) {
  code: 13,
  details: 'Received RST_STREAM with code 2',
  metadata: Metadata { internalRepr: Map {}, options: {} }
}
Error: 8 RESOURCE_EXHAUSTED: Bandwidth exhausted
    at Object.callErrorFromStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call.js:30:26)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client.js:175:52)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:341:141)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:304:181)
    at Http2CallStream.outputStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:116:74)
    at Http2CallStream.maybeOutputStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:155:22)
    at Http2CallStream.endCall (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:141:18)
    at ClientHttp2Stream.<anonymous> (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:403:22)
    at ClientHttp2Stream.emit (events.js:219:5)
    at emitErrorCloseNT (internal/streams/destroy.js:76:8) {
  code: 8,
  details: 'Bandwidth exhausted',
  metadata: Metadata { internalRepr: Map {}, options: {} }
}
Error: 8 RESOURCE_EXHAUSTED: Bandwidth exhausted
    at Object.callErrorFromStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call.js:30:26)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client.js:175:52)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:341:141)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:304:181)
    at Http2CallStream.outputStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:116:74)
    at Http2CallStream.maybeOutputStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:155:22)
    at Http2CallStream.endCall (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:141:18)
    at ClientHttp2Stream.<anonymous> (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:403:22)
    at ClientHttp2Stream.emit (events.js:219:5)
    at emitErrorCloseNT (internal/streams/destroy.js:76:8) {
  code: 8,
  details: 'Bandwidth exhausted',
  metadata: Metadata { internalRepr: Map {}, options: {} }
}
Error: 8 RESOURCE_EXHAUSTED: Bandwidth exhausted
    at Object.callErrorFromStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call.js:30:26)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client.js:175:52)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:341:141)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:304:181)
    at Http2CallStream.outputStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:116:74)
    at Http2CallStream.maybeOutputStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:155:22)
    at Http2CallStream.endCall (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:141:18)
    at ClientHttp2Stream.<anonymous> (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:403:22)
    at ClientHttp2Stream.emit (events.js:219:5)
    at emitErrorCloseNT (internal/streams/destroy.js:76:8) {
  code: 8,
  details: 'Bandwidth exhausted',
  metadata: Metadata { internalRepr: Map {}, options: {} }
}
Error: 13 INTERNAL: Received RST_STREAM with code 2
    at Object.callErrorFromStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call.js:30:26)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client.js:175:52)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:341:141)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:304:181)
    at Http2CallStream.outputStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:116:74)
    at Http2CallStream.maybeOutputStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:155:22)
    at Http2CallStream.endCall (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:141:18)
    at ClientHttp2Stream.<anonymous> (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:403:22)
    at ClientHttp2Stream.emit (events.js:219:5)
    at emitErrorCloseNT (internal/streams/destroy.js:76:8) {
  code: 13,
  details: 'Received RST_STREAM with code 2',
  metadata: Metadata { internalRepr: Map {}, options: {} }
}
Error: 13 INTERNAL: Received RST_STREAM with code 2
    at Object.callErrorFromStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call.js:30:26)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client.js:175:52)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:341:141)
    at Object.onReceiveStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:304:181)
    at Http2CallStream.outputStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:116:74)
    at Http2CallStream.maybeOutputStatus (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:155:22)
    at Http2CallStream.endCall (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:141:18)
    at ClientHttp2Stream.<anonymous> (/home/ec2-user/.nvm/versions/node/v13.3.0/lib/node_modules/node-firestore-import-export/node_modules/@grpc/grpc-js/build/src/call-stream.js:403:22)
    at ClientHttp2Stream.emit (events.js:219:5)
    at emitErrorCloseNT (internal/streams/destroy.js:76:8) {
  code: 13,
  details: 'Received RST_STREAM with code 2',
  metadata: Metadata { internalRepr: Map {}, options: {} }
}
All done 🎉

Some data is imported into the firestore database, but other data is not imported.

Steps to reproduce the behavior

Using version 1.1.0

Running command /home/ec2-user/.nvm/versions/node/v13.3.0/bin/firestore-import -a ~/pc-test-firebase-adminsdk-5relp-daf7fa32cf.json -b backup-202005251534.json

The Firestore project is using the Blaze plan

[jontybrook]

I'm also facing this issue

[3rcok]

I had the same issue

[kisonay]

anyone able to resolve this?

[MdFarhanRaja]

I'm also facing same. Please please please help...

[ghost]

Same please help

[tilmanschweitzer]

I think this issue can be fixed by limiting the concurrency. If a limitation is necessary seems to depend on the content (depth, collection size etc). I'll open a PR later.

[nonoumasy]

Just wondering if this issue has been resolved. I'm also getting the 'Bandwidth exhausted' error. cheers