RafaPolit
commented
5 years ago We have been discussing some of these approaches ourselves. But we have changed a lot how the migrations are run. Now we store the 'last migration to have been run against the current DB' within the DB. And the migration process parses the files to determine which is the last 'migration' found in the code. With that info, only the migrations that are missing are run (if any) and in the correct order.
So... several things to point out:
- This actually checks what is the current 'data version' (let's not call it database version as it could mean the actual MongoDB version which should be exactly the one we request for in the dependencies)
- This checks the current 'code version' in terms of migrations
- It runs only the migrations that are required
So, its perfectly safe to call on migrate every time you are going to start the server instance without any nasty side effects.
Still, we have been discussing more complex scenarios like clients running older code that could potentially interact wrongly with the API and, also, checking that the correct dependencies versions are installed before deploying a server instance. That would involve checking for MongoDB, Elastic and some other minor dependencies.
I agree this is an important thing to have, but, in the meantime, just calling on "yarn migrate" will achieve what you want (mostly). The key thing here is how to know if the previous "yarn migrate" requires or not a reindex? That actually requires some thought... we have been doing it 'manually': we know there is a migration that requires reindexing, so we are going to run the reindex as well. Obviously, there is no downside on calling the reindex after running the migrations either... but it could be time and resource consuming, so I understand the need / want to avoid doing an unnecessary reindexing each time. But it's the only way to be really sure everything works.
But we should be able to tag a migration like 'requires reindex' or even a code change that requires reindexing. So yes, that requires attention.
Do you think this helps a bit on your flow? Or do I missinterpreted the reasoning behind what you are asking for?
Best regards, Rafa.
fititnt
txau
See also:
2041
This issue can take some months to be somewhat fully automated (at least for the uwazi-docker version) but at the very basic, beyond to make user aware of potential different verions of database vs the code that wants to run that database (in this case, one screen on administrator could be sufficient) there is some ideas for take in consideration to be able to run as command line. Note that this is just a draft idea, and some commands maybe could be simplified than what is here or some of these are just redundant:
Explanation:
If, in a very, very simple way, that can be parsed in a bash script, like return 0 if no migration is need, return 1 or more if migrations are (for sure) need and -1 if something is very, very wrong (for example, a lower version of uwazi code is trying to run a higher version of database, or maybe even that there is no installed database) this could be automated in a dockerized version of uwazi.
And yes, I'm aware that potentially running migrations could be go wrong, but at the very basic, at least know that migrations are need (note again: very, very basic checking) could be useful to at least force someone who is running a dockerized version run some extra parameter, like "--force" to solve the issues manually.