Could not write collection data into - Error: size mismatch

[truongtv22]

I try to create a simple example following your instructions. When I try to run, the error is like this.

could not write collection data into: type: local, path: ./backup.tar due to error: Error: size mismatch removing: "type: local, path: ./backup.tar" because an error was thrown during the transfer

[lihaibh]

@truongtv22 can you please try do the following: on your source connector add the property:

 assource: {
      collections: ['collection1'],
    },

in order to backup a specific collection. I want to see if this anything to do with a specific collection that you are trying to backup. it should look like the following:

    const mongo_connector = new MongoDBDuplexConnector({
        connection: {
            uri: `mongodb://<username>:<password>@<hostname>:<port>`,
            dbname: '<database-name>',
        },
       assource: {
          collections: ['collection1'],
        },
    });

This error occasionally occurs when there is a mismatch between the actual amount of bytes you write for a collection and the collection size in mongo.

[truongtv22]

Yes, i have tried the following example

import { MongoTransferer, MongoDBDuplexConnector, LocalFileSystemDuplexConnector } from 'mongodb-snapshot';

async function main() {
  const mongo_connector = new MongoDBDuplexConnector({
    connection: {
      uri: 'mongodb://localhost:27017',
      dbname: 'test',
    },
    assource: {
      collections: ['users'],
    },
  });
  const localfile_connector = new LocalFileSystemDuplexConnector({
    connection: {
      path: './backup.tar',
    },
  });

  const transferer = new MongoTransferer({
    source: mongo_connector,
    targets: [localfile_connector],
  });

  for await (const { total, write } of transferer) {
    console.log(`remaining bytes to write: ${total - write}`);
  }
}

main();

[lihaibh]

@truongtv22 it happens for any collection you choose? what mongodb version are you using? what version of the library are you using?

[truongtv22]

It happens with any collection

MongoDB server version: 4.4.1 mongodb-snapshot: "^1.2.0"

[lihaibh]

@truongtv22 i did not test it with mongo v4+. only with v3 i will try and get back to you. Maybe i will have to create a connector for mongo v4 because im using a db driver in v3

[truongtv22]

Thanks for your job

[lihaibh]

@truongtv22 one last thing can you configure your target connector as follows:

  const localfile_connector = new LocalFileSystemDuplexConnector({
    connection: {
      path: file_full_name,
    },
    astarget: {
      collections: [],
      metadatas: ['users'],
    },
  });

in order to write only metadata for users collection.

[truongtv22]

Do you have a way to write the data?

[lihaibh]

@truongtv22 I found the error. im openning a fix.

[lihaibh]

@truongtv22 please use version 1.2.1 and let me know if it works for you.

[truongtv22]

Great, it works for me