dalenguyen / firestore-backup-restore

NPM package for backup and restore Firebase Firestore
https://firebaseopensource.com/projects/dalenguyen/firestore-backup-restore/
MIT License
495 stars 84 forks source link

Build and deployment failed due to an error. #144

Closed RamDivrania closed 2 years ago

RamDivrania commented 2 years ago

I've no exact title for this error, but it's not working for me.

node_modules/firestore-export-import/node_modules/@google-cloud/firestore/types/firestore.d.ts(117,29): error TS1005: ']' expected.
node_modules/firestore-export-import/node_modules/@google-cloud/firestore/types/firestore.d.ts(117,48): error TS1005: ';' expected.
node_modules/firestore-export-import/node_modules/@google-cloud/firestore/types/firestore.d.ts(117,50): error TS1109: Expression expected.
node_modules/firestore-export-import/node_modules/@google-cloud/firestore/types/firestore.d.ts(117,51): error TS1109: Expression expected.
node_modules/firestore-export-import/node_modules/@google-cloud/firestore/types/firestore.d.ts(2278,1): error TS1128: Declaration or statement expected.
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! functions@ build: `tsc`
npm ERR! Exit status 2
npm ERR! 
npm ERR! Failed at the functions@ build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2022-09-15T10_33_21_750Z-debug.log
Error: functions predeploy error: Command terminated with non-zero exit code2
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: exit code 1

Below are a few package versions I'm using atm..

 firebase-tools@10.9.2
    "typescript": "^4.0.3"
    "firebase-admin": "^9.2.0",
    "firebase-functions": "^3.13.2",
    "firebase-functions-helper": "^0.8.0",
    "firestore-export-import": "^1.2.1",

Tried upgrade, downgrading and other stuff but nothing is working.

firebase deploy --project staging --token $FIREBASE_TOKEN

RamDivrania commented 2 years ago

I'm using this package firestore-export-import with typescript inside google cloud function and getting errors after > tsc

RamDivrania commented 2 years ago

can't we use firebase-admin and firestore-export-import together? This problem occurs only when I try to import/initialise both. as my cloud function already using firebase-admin.

node_modules/@google-cloud/firestore/types/firestore.d.ts:25:1 - error TS6200: Definitions of the following identifiers conflict with those in another file: DocumentData, UpdateData, Firestore, GeoPoint, Transaction, BulkWriter, WriteBatch, SetOptions, WriteResult, DocumentReference, DocumentSnapshot, QueryDocumentSnapshot, OrderByDirection, WhereFilterOp, Query, QuerySnapshot, DocumentChangeType, CollectionReference, FieldValue, FieldPath, Timestamp, v1beta1, v1, OK, CANCELLED, UNKNOWN, INVALID_ARGUMENT, DEADLINE_EXCEEDED, NOT_FOUND, ALREADY_EXISTS, PERMISSION_DENIED, RESOURCE_EXHAUSTED, FAILED_PRECONDITION, ABORTED, OUT_OF_RANGE, UNIMPLEMENTED, INTERNAL, UNAVAILABLE, DATA_LOSS, UNAUTHENTICATED, FirebaseFirestore

25 declare namespace FirebaseFirestore {
   ~~~~~~~

  node_modules/firestore-export-import/node_modules/@google-cloud/firestore/types/firestore.d.ts:23:1
    23 declare namespace FirebaseFirestore {
       ~~~~~~~
    Conflicts are in this file.

node_modules/@google-cloud/firestore/types/firestore.d.ts:168:5 - error TS2374: Duplicate index signature for type 'string'.

168     [key: string]: any; // Accept other properties, such as GRPC settings.
        ~~~~~~~~~~~~~~~~~~~

node_modules/firestore-export-import/node_modules/@google-cloud/firestore/types/firestore.d.ts:23:1 - error TS6200: Definitions of the following identifiers conflict with those in another file: DocumentData, UpdateData, Firestore, GeoPoint, Transaction, BulkWriter, WriteBatch, SetOptions, WriteResult, DocumentReference, DocumentSnapshot, QueryDocumentSnapshot, OrderByDirection, WhereFilterOp, Query, QuerySnapshot, DocumentChangeType, CollectionReference, FieldValue, FieldPath, Timestamp, v1beta1, v1, OK, CANCELLED, UNKNOWN, INVALID_ARGUMENT, DEADLINE_EXCEEDED, NOT_FOUND, ALREADY_EXISTS, PERMISSION_DENIED, RESOURCE_EXHAUSTED, FAILED_PRECONDITION, ABORTED, OUT_OF_RANGE, UNIMPLEMENTED, INTERNAL, UNAVAILABLE, DATA_LOSS, UNAUTHENTICATED, FirebaseFirestore

23 declare namespace FirebaseFirestore { 
dalenguyen commented 2 years ago

Looks like it relates to @google-cloud packages. Do you have a sample repo that I can have a look at?

RamDivrania commented 2 years ago
import * as admin from "firebase-admin";
import * as firestoreExportImport from "firestore-export-import";

admin.initializeApp();
admin.firestore().settings({ ignoreUndefinedProperties: true });
firestoreExportImport.initializeFirebaseApp(admin.credential.applicationDefault());

// The bucket where the backup will be stored.
const backup_bucket = "gs://backup-bucket";

exports.scheduledFirestoreExport = functions
  .region("europe-west1")
  .pubsub.schedule("10 * * * *")
  .onRun((context) => {
    console.log("scheduledFirestoreExport trigered at => ", new Date());
    if (process.env.GCLOUD_PROJECT !== "prod-plattform") {
     **// Backup using firestore-export-import**
      console.log("Starting json backup of database collections \n\n ");
      firestoreExportImport.backups([])
        .then((collections) => {
          console.log("Data fetched for collection \n\n ");
          console.log(JSON.stringify(collections, null, 2));
          console.log("\n\n Backup finished");
        })
        .catch((e) => {
          console.log(
            "Error while creating backup of database via firestore-export-import => ",
            JSON.stringify(e, null, 2)
          );
        });
      return;
    }

     **//backup using standard firebase export which doesn't export as JSON**
    const databaseName = client.databasePath(process.env.GCLOUD_PROJECT, "(default)");
    // First: List all existing collection ids
    admin
      .firestore()
      .listCollections()
      .then((result) => {
        const collections = [];

        for (const collection of result) {
          console.log(`Adding collection to the backup list: `, JSON.stringify(collection.id, null, 2));
          collections.push(collection.id);
        }

        return client
          .exportDocuments({
            name: databaseName,
            outputUriPrefix: backup_bucket,

            // For collections to be restored individually, they must be specified individually with their collection id in the export.
            collectionIds: collections,
          })
          .then((responses) => {
            const response = responses[0];
            console.log(`Operation Name: ${response["name"]}`);
            return response;
          })
          .catch((err) => {
            console.error(err);
            throw new Error("Export operation failed");
          });
      });
  });
dalenguyen commented 2 years ago

Hi @RamDivrania, you should initialize app once. You can use this only.

// admin.initializeApp(); <-- should be removed.

firestoreExportImport.initializeFirebaseApp(admin.credential.applicationDefault());
dalenguyen commented 2 years ago

You also want to add "skipLibCheck": true to your tsconfig.json.

https://www.typescriptlang.org/tsconfig/#skipLibCheck

RamDivrania commented 2 years ago

Hello @dalenguyen, You're right, But as I mentioned earlier, this function is already using firebase-admin for other purposes and I need both packages here because I want normal backup + JSON backup.

dalenguyen commented 2 years ago

@RamDivrania have you tried to "skipLibCheck": true?

RamDivrania commented 2 years ago

That is working like a charm, and deployment worked! Thanks 😊 But I'm getting other errors after deploying the same code.

2022-09-22 18:30:03.506 ISTscheduledFirestoreExporth6sczs2oi736 SyntaxError: Unexpected token '.' at wrapSafe (internal/modules/cjs/loader.js:915:16) at Module._compile (internal/modules/cjs/loader.js:963:27) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10) at Module.load (internal/modules/cjs/loader.js:863:32) at Function.Module._load (internal/modules/cjs/loader.js:708:14) at Module.require (internal/modules/cjs/loader.js:887:19) at require (internal/modules/cjs/helpers.js:74:18) at Object.<anonymous> (/workspace/node_modules/firestore-export-import/node_modules/firebase-admin/lib/app/lifecycle.js:25:24) at Module._compile (internal/modules/cjs/loader.js:999:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10) at Module.load (internal/modules/cjs/loader.js:863:32) at Function.Module._load (internal/modules/cjs/loader.js:708:14) at Module.require (internal/modules/cjs/loader.js:887:19) at require (internal/modules/cjs/helpers.js:74:18) at Object.<anonymous> (/workspace/node_modules/firestore-export-import/node_modules/firebase-admin/lib/app/index.js:22:19) at Module._compile (internal/modules/cjs/loader.js:999:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10) at Module.load (internal/modules/cjs/loader.js:863:32) at Function.Module._load (internal/modules/cjs/loader.js:708:14) at Module.require (internal/modules/cjs/loader.js:887:19) at require (internal/modules/cjs/helpers.js:74:18) at Object.<anonymous> (/workspace/node_modules/firestore-export-import/dist/index.js:4:725)
SyntaxError: Unexpected token '.' at wrapSafe (internal/modules/cjs/loader.js:915:16) at Module._compile (internal/modules/cjs/loader.js:963:27) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10) at Module.load (internal/modules/cjs/loader.js:863:32) at Function.Module._load (internal/modules/cjs/loader.js:708:14) at Module.require (internal/modules/cjs/loader.js:887:19) at require (internal/modules/cjs/helpers.js:74:18) at Object.<anonymous> (/workspace/node_modules/firestore-export-import/node_modules/firebase-admin/lib/app/lifecycle.js:25:24) at Module._compile (internal/modules/cjs/loader.js:999:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10) at Module.load (internal/modules/cjs/loader.js:863:32) at Function.Module._load (internal/modules/cjs/loader.js:708:14) at Module.require (internal/modules/cjs/loader.js:887:19) at require (internal/modules/cjs/helpers.js:74:18) at Object.<anonymous> (/workspace/node_modules/firestore-export-import/node_modules/firebase-admin/lib/app/index.js:22:19) at Module._compile (internal/modules/cjs/loader.js:999:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10) at Module.load (internal/modules/cjs/loader.js:863:32) at Function.Module._load (internal/modules/cjs/loader.js:708:14) at Module.require (internal/modules/cjs/loader.js:887:19) at require (internal/modules/cjs/helpers.js:74:18) at Object.<anonymous> (/workspace/node_modules/firestore-export-import/dist/index.js:4:725)
dalenguyen commented 2 years ago

It's probably you have an older version of Node. Here is a similar error https://github.com/HashLips/hashlips_art_engine/issues/424