Create revert installers

[WebDevJL]

See the updaters as a base: https://github.com/Puzzlout/TrekToursInstaller/blob/master/Updaters.Usage.md

The usage is when we release a v2 of the API for instance and we see that sometimes is wrong. We want to revert to v1. Also we need to make sure that if a database update was brought with v2, reverting to v1 _MUST NOT_ impact the application. I think that is possible if we code properly following the guidelines of this wiki: https://github.com/Puzzlout/TrekToursApi/wiki/Versioning

The task is to revert from v2 to v1 with those actions:

• switch to v1 branch
• run the Symphony warmup commands (similar to the updaters after the git checkout
• delete the v2 branch since the fix will be released in a v3

We will need a local installer and c9 installer like the updater scripts.

Update this file after you are done with the procedure: https://github.com/Puzzlout/TrekToursInstaller/blob/master/Reverters.Usage.md

Let me know if you have questions. I will answer them no later than Monday (I have very little time until then... Sorry). You start and experiment the part with the DB versioning.

[WebDevJL]

@damirius This is a reply following up on your email:

I'll try to come up with something, we essentially need to reset the db changes and switch back to chosen version branch, idea is to run it like this "bash revert.sh v1"? As for versioning that we have in place, it's used little different than that. In our entities we can add annotation to the field so we can tell which field is used with what version, for example annotation "@Since("2.0")" will display that field only in v2 calls not in versions before that one. We can also add "@Version" annotation in our controller actions, so that two actions represent same endpoint but for different versions, for example if we call /customerinforequests?version=1 we will call one action and if we call /customerinforequests?version=2 we call another action. So versioning system in place is more in place so we have some sort of backward compatibility and we have multiple version of the api in the place in the same time.

So I think the databases changes do not need to be reverted as the code calling it will not exist and is versioned.

So let's say we add new tables or new columns to existing tables, after the git checkout to older version branch, those tables will not be used and the columns will have default values so the older version of the code can still insert rows on modified tables.

Then, we can either remove them using doctrine commands (possible?) or using a simple SQL drop script.

Have you looked at https://github.com/Puzzlout/TrekToursInstaller/issues/3 ? I have detailed the steps to take in the reverting. Taking into account the description of the task, is the use of annotation @Version or @Since making a difference?

The only case that I think would fail is when we change an existing column of an existing table. Except if we make sure the modification work in v1 and v2.

Yes, the revert command would be bash revert.sh 1.0.0.xxx (no "v" suffix).

[damirius]

@WebDevJL

OK, so main reason for the revert script is to revert the database, since with git checkout we can change all the code on the working branch, but DB stays the same right?

Let's say we add new field called test to the CustomerInfoRequest entity, developer adding that will also add getter and setter for that field or generate it with php bin/console doctrine:generate:entities ApiBundle for example. In any case we have updated entity with new test field. We run php bin/console doctrine:schema:update --force which will execute necessary queries and test field will be shown as column in our customerinforequests table in our MySql DB. We see that we missed some critical bug in that version and we decide we want to revert all the code to some earlier stable release. We just checkout (we can pull too) to that stable branch and run php bin/console doctrine:schema:update --force, since entity doesn't contain test field, that command will remove that test field from the database. Of course all the data will be removed from that column also, but if we are reverting we want to do that in any case.

OK so we are left with that problem with renaming columns, that's where built in versioning comes into place. Let's say we have first_name column, like we do in our entity. For some reason we want to change it from varchar into text (in doctrine from string to text type) in our 1.0.0.670 version. That one shouldn't cause too much problems, since data will be left intact if I'm not mistaken. But if we want to do it the right way we can do this:

/**
* @var string
*
* @ORM\Column(name="first_name", type="string", length=100)
* @Serializer\Groups({"list", "details"})
* @Until("1.0.0.669")
*/
private $firstName;

That's the same as we currently have it listed, just with @Until annotation with last version that uses it. Now we add totally new field to the database.

/**
* @var string
*
* @ORM\Column(name="first_name_2", type="text")
* @Serializer\Groups({"list", "details"})
* @Since("1.0.0.670")
* @SerializedName("first_name")
*/
private $firstName2;

So we have totally new field called firstName2 or first_name_2 in db which is text type. We tell the serializer that it's used since 1.0.0.670 version and that it should be display like first_name. So @SerializedName can be used to display new field like the old ones without the need to delete old one. Now of course if we would like to reuse the data also we would probably need to migrate the data from old one to the new one, but this isn't in case of reverting but in case of new versions.

So conclusion is we can revert back to earlier releases with same update script we are using to switch to newer ones, schema:update will also revert back the db according to the code we checked out to.

[WebDevJL]

@damirius

At first, I thought that doing a checkout of an older branch version would be enough to revert but now I see your point and I think you describe the true goal of the revert.

So we never modify the old column first_name but we add a new one to which the code is pointing to from v1.0.0.670. That would imply that we need to copy the data from first_name to first_name_2. Correct?

However, what happens if we have v1.0.0.670 released for a week and we see a problem. New data will have been inserted during that week and we may not be able to copy data from first_name_2 to first_name since the db type is larger in first_name_2. Correct? I think the only solution would be a hotfix release of the bug and only that.

Also, we cannot change PK or FK very easily once done or have you had experience with such scenario?

And a final question: can we purge the old column in the code and the db at one stage?

Thanks for the analysis.

[WebDevJL]

@damirius What's the progress of this task? Thanks.

[damirius]

@WebDevJL

Well as we concluded, our updaters already work as revert scripts, since we can specify some previous releases to "update" too and doctrine will update schema accordingly.

Yes some stuff we can't revert, if we go for example from varchar 30 in length to varchar 50 for name and we add some data with 30+ characters and then we go back to 30 length varchars we will purge that data since we can't store it.

Changing PK/FK is also critical since we can't just change those types of columns back and forth.

We can purge old columns in some version just by removing completely column from our entity map class when we want it purged.

So like always we should be smart about db design and think of as much things as we can when we design it so we don't run in such problems, especially with PKs/FKs.

Thanks

[WebDevJL]

​@damirius said: So like always we should be smart about db design and think of as much things as we can when we design it so we don't run in such problems, especially with PKs/FKs.

Indeed, it is the best reverter solution ;)

I will talk to the client about the DB type size update to see what he thinks. In that particular case, I think we shouldn't have to revert this change since, if there is a problem, I assume it would be in the code rather than in the DB. What do you think?

Jérémie Litzer CEO @ Puzzlout http://puzzlout.com

On 5 August 2016 at 10:19, damirius notifications@github.com wrote:

@WebDevJL https://github.com/WebDevJL

Well as we concluded, our updaters already work as revert scripts, since we can specify some previous releases to "update" too and doctrine will update schema accordingly.

Yes some stuff we can't revert, if we go for example from varchar 30 in length to varchar 50 for name and we add some data with 30+ characters and then we go back to 30 length varchars we will purge that data since we can't store it.

Changing PK/FK is also critical since we can't just change those types of columns back and forth.

We can purge old columns in some version just by removing completely column from our entity map class when we want it purged.

So like always we should be smart about db design and think of as much things as we can when we design it so we don't run in such problems, especially with PKs/FKs.

Thanks

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Puzzlout/TrekToursInstaller/issues/3#issuecomment-237785716, or mute the thread https://github.com/notifications/unsubscribe-auth/ADV_mlx0_-A4YCyk8xTLjJTAqvHbDQjuks5qcvIEgaJpZM4JS8WG .

[damirius]

@WebDevJL

I agree. Also reverting something should mean that there's some critical bug, if we go through all that testing we will probably catch it before, if not we can also try hotfixing it and releasing new version which should fix those things in the code. Like you said most of the time it's problem with the code if we design the DB right. For example creating varchars with more length than needed is honestly better solution than to cap it at some low value then trying to change it later on. Or setting some field as varchar instead of int (let's say phone number) is better if we are thinking of adding special signs in it. Thinking ahead is always good when working with DBs :)