seppevs / migrate-mongo

A database migration tool for MongoDB in Node
MIT License
928 stars 165 forks source link

Issue probably with the connection or connection pool #26

Closed stherrienaspnet closed 6 years ago

stherrienaspnet commented 6 years ago

ISSUE DESCRIPTION: When we run the migration the first time, it is working fine. But when running for the second time after having deleted the database i got this error.

(node:3403) UnhandledPromiseRejectionWarning: MongoError: a collection 'test.physical-type' already exists.

REPRODUCTION: 1- git clone this repository https://github.com/stherrienaspnet/migrate-mongo-issue.git 2- enter command: 'npm run deploy' 3- check the db test, you should see correct result 4- delete the test db using compass or Robo 3T 5- enter command: 'npm run deploy', you should see the error message.

ADDITIONAL INFO: I added the file deployme.js to use only the native mongodb driver and running the same script and deleting the test db several times and this is working perfectly.

Hope you will find the route cause, because I would prefer to use your migration project instead of creating mine from scratch.

Many thanks

seppevs commented 6 years ago

I don't think that's a migrate-mongo issue.

The error test.physical-type' already exists indicates the collection 'physical-type' already exists. Are you sure you have deleted your existing data?

I also don't understand your setup, it looks way too complicated to me.

For example:

  1. I see that your migration files are in the directory 'scripts/sprint001', while your config indicates they are in the dir 'migrations'.
  2. Why did you rename & move the generated migration files? It's not something I supported. Plus, the generated 'timestamp' prefix is important: it is used to determine the order of your migrations.
  3. I have no idea what npm run deploy does, because you have not checked in a package.json file in your migrate-mongo-issue git repo.

Please follow the Quickstart section of the README, and use migrate-mongo as intended.

stherrienaspnet commented 6 years ago

Hello, I just push the file package.json sorry for that. If you make a pull you will see it clearly, the predeloy.js will just take script in the folders and move them into migrations folder and they will be in alpha order. I really deleted manually the database. If you try it you will probably see what I see. Just exec npm run deploy and delete the test bd and exec npm run deploy again you should see that strange behaviour.

seppevs commented 6 years ago

I currently don't have the time to reproduce your issue, but I'm 100% sure it's not an issue with migrate-mongo: The error is clearly stating that the collection test.physical-type already exists (when your migration is trying to create that collection).

EDIT: I've just spotted a serious flaw in your migration script: https://github.com/stherrienaspnet/migrate-mongo-issue/blob/master/scripts/sprint001/script001-add-physical-type.js => You did not call next() callback correctly. You need to call it when all your I/O operations are done. In your script, you did not provide a callback to the createCollection and insertMany functions (or used their Promise counterpart).

Please read how i/o works in Node.js. I'm afraid I don't have the time to help you with that.

stherrienaspnet commented 6 years ago

Thanks :) using async(db, next), await and next to the latest io operation finally resolve it :)

seppevs commented 6 years ago

Great, good luck!