search

seppevs / migrate-mongo

A database migration tool for MongoDB in Node
MIT License
931 stars 166 forks source link

Using session transactions in multiple migrations #286

Open ghost opened 3 years ago

ghost commented 3 years ago

Hello !

First of all, thanks for this cool lib @seppevs !

I would like to know if it is possible to use a session transaction in multiple migrations. My goal is to allow rollback of multiple migrations in case of error.

A use case for example : My current migration version is 3 and I have 4 migrations to go up. I run a mongo-migrate up, but the 3rd migration fails.

I would like to roll back not only the 3rd but also the 1st and 2nd using the same transaction. After fail, we would the db would stay at the state of version 3.

Then, it would be easier and safer to sync my migrations with my code deployments.

This could also be done by using down scripts but I will have to track the initial version and run down after failure until initial state is retrieved. I don't know which solution is doable and which is the trickiest ?

Any idea ?

Thanks

Best, Antoine

ftreguer commented 3 years ago

Hello, I had the same issue than you, I wanted to rollback all migrations from the same block of execution. I've implemented the rollback of a group of migration scripts and proposed it in a pull request. The code is here: https://github.com/seppevs/migrate-mongo/pull/331 If you have any feedback, or improvements, we can discuss :)

ghost commented 3 years ago

Hello Fabien,

Thanks for your answer and sorry for the delay.

I started reviewing your PR, it looks quite interesting.

I saw that you implemented a -b option to the down command. If I understand well that means you have to trigger the rollback manually, by launching a down -b and it will rollback the last migrated block (based on last timestamp). That is a very good option to have in that lib.

However, I am not sure we are facing the exact same situation here.

This is the workflow I would like to get :

  1. migrate up is run (potentially with multiple scripts in sequence)
  2. One of the scripts fails
  3. Automatic rollback to the initial state (before the migrate up run) is triggered by the failure

That is why I suggested to propagate a mongo session into a group of scripts (such as the migration block as you call it), to rollback in case of a failure. Rollbacking using down as you implemented can be an option too, but my need was to trigger it automatically after an error occurs.

This would allow to make clean continuous deployments in Heroku for example.

Do you have a community slack or any other app so that we can discuss about it ? :)

I could help you to vaidate and improve your implementation.

Best

Antoine

ftreguer commented 3 years ago

Hi Antoine, thank you for your response. I needed to rollback several migration scripts ran in a same migration if an error occurred during migration or during an other step of my application's deployment. I use a deployment flow with several hooks, so if any step of deployment throws an error I have to rollback migration and stop deployment. That's why I prefer to use down scripts with a block of scripts.

Fabien

ghost commented 3 years ago

Ok so in case of an error, you juste run this other command, that's it ?

The flow is in two steps then.

Ok that sounds good.

But what do you think about the idea of mongo sessions to avoid writing on the DB if an error occurs ? Rollbacking using down and using transactions are two different situations : without using transactions, the data is up on production, and that can be a problem before code deployment.

ghost commented 3 years ago

FYI, I think I will propose an improvement of the up command also.

Currently, up executes all forward migration scripts. It lacks a function that targets a specific script, or even up running a script one by one. This kind of feature is available in other libs in SQL (sequelize for example) or even mongoose migrate.

It can be useful to have more control on up and down commands.