Add Tags Feature

[xrow]

I want to have a feature like this:

http://docs.ansible.com/ansible/playbooks_tags.html

Tags could be install and update.

and you could run:

app/console kaliop:migration:migrate -n --skip-tags "install"

or

app/console kaliop:migration:migrate -n --tags "upgrade"

Would that be accepted?

[gggeek]

Hello.

If all of your migrations are stored in a single place, such as bundle or a single folder, you can already achieve the same result by grouping them into subfolders and using the --path option. Eg:

ka:mi:migrate --path=ezpublish/migs/install
ka:mi:migrate --path=ezpublish/migs/upgrade
ka:mi:migrate --path=ezpublish/migs (this is in fact missing but should be easily doable with a recursive loader)

The fact that migrations never get executed twice also means that in standard deployment scenarios there is no risk in keeping all migrations together and running them multiple times.

In short: while I am not completely averse to this feature, I don't see a huge benefit to it - maybe you can explain better your usecases?

Last but not least, I wouldn't like to add too much extra complexity to the core - supporting workflows as extended-migrations will already stretch the base design to its acceptable limits in terms of complexity and bloat. In other words, if tags go in, they should probably be implemented in the loader subsystem or via some other hook, not by adding stuff to the base classes.

[xrow]

Hi @gggeek,

The are not stored in a single place. They are stored per bundle. We want to do a seperation of concerns.

An alternative to this would be a "skip" feature. That would mark things to be not executed on systems that on which a install migration had been already applied.

How about that?

[gggeek]

Hello. The 'tag version as skipped' feature is already in place, but probably not 100% user-friendly or complete for your needs. If you're happy top play around with it, I'll gladly work on any improvements you might need.

[xrow]

Hi Gaetano,

can you pin point me to the documentation or code of this 'tag version as skipped' feature. I can`t find it somehow.

[gggeek]

It should be mentioned in the whatsnew file. It is an option of the 'migration' command

[xrow]

I reviewed this function. This is not exactly what I am after. I want to split my migrations into install migrations and update migrations. Sometimes install migration do not need to get run since they tables already exist or install had been performed manually before, but the skip option can be only applied, if the file already exits in the installation. In an automated deployment scenario I can`t mark those skip files ahead in time so they would always get executed. So the scenario i want to cover is some automatic install/setup task that can compare the schema before and after.

What is you proposal for this kind of use?

Also it is worth mentioning that of course each install migrations shall be placed in it`s corresponding bundle.

[gggeek]

Random thoughts:

• automatic convergence to a desired state is a very hard thing to do. It is definitely not in the road map for the migration bundle to be able to define a target a 'state' in a declarative way and let the system work out the correct steps to take to get there. All it will support is imperative step definitions (on a tangent: my personal experience with systems which try to converge to a desired state such as eg. Chef is that they should burn in hell :-P)

• as for tagging migrations skipped ahead-of-time: you can mark them as skipped as soon as they are on disk. Maybe you could write a deployment step that happens after code has been copied to the target server but before migrations are executed and checks for the tables to exist already? Or you can add to your 'creation' migrations a 1st custom step which throws a 'migration aborted' exception if the desired table exists already (this is a new exception recently added for this very scenario)

[xrow]

Hmm maybe I catch the return code of the SQL and check for the type of error. If the type of the error is table exists we assume that the it is not nessecary to run it.

[gggeek]

Note that since version 5,0 and maybe even 4,x you can add an if condition to every migration step. This makes it somewhat easier to do conditional migrations (eg. load first entity x, if it exists skip next step which is entity creation and go to step 3 which is entity update)

[gggeek]

Other random thoughts:

• as soon as ver 5.13 is out, you will be able to run sql queries in yaml migration steps and set data from the result set to references
• you will also be able to resolve references in any sql query to be executed as migration step Maybe those can help you writing 'flexible migrations'...

Other extension points that you could use:

• use a customized migration definition loader that is smart enough to figure which migrations to 'skip'
• write a custom service which can be run using the Service migration step. This can check if the desired tables exist running complex business logic and set the reference value that is used as switch for the 'skip' condition
• same but using a shell script and invoking it using the Process migration step

Of course neither of these options is a turnkey solution for your problem, but as I stated above, being able to 'generate a db diff from current table definitions and desired schema' is not a target goal. Maybe using Doctrine Migrations could help with that though...

Let me know if you think this is sufficient (or if you have found a way to achieve what you wanted) and I'll close the ticket...

[gggeek]

Other enhancements have been added in 5.13 that allow to achieve some kind of 'upsert' functionality:

As stated above, for creation of custom tables, you can do a migration with:

• step 1: sql/select: load from db dictionary the table named XXX, create a reference found to attribute count
• step 2: sql/exec: create table iff reference:count equals 0

For creation of eg. content types, you can do a migration with:

• step 1: content_type/load: match the CT with identifier XXX, use match_tolerate_misses: true, create a reference found to attribute count
• step 2: content_type/create: create table iff reference:count equals 0

[gggeek]

Long time without feedback. Closing