[Discussion] Updates and patch files

[zebrahosting]

There is a tremendous effort to improve and extend Yeti but at the same time the update cycle takes longer and my stable 2.0 version is more or less unusable. Wouldn't it be a good idea to start working with Update files (adding new options) and patch files (correcting bugs). I would love to start using 2.1 but it still needs bug fixing. Some of the bugs will only surface after many people are using it and we tend to get in the vTiger 'trap' that update cycles take so long. If we use dev versions we cannot upgrade properly. I guess it is a coders nightmare to go this way but from a users point of few it seems to make sense.

Copying over the files should never be a problem but tracking down what was changed in DBtables and getting those changes in the local system seems harder. How is it done with the demo / test systems at Yetiforce? Running some scripts? Guess it overwrites everything to back to default while we use a lot of extra custom fields for example. Food for thoughts.

Curious how others think about this and what the practical options are.

[diderich]

My wishlist would be a tool / functionality that allows to export the content (customization + data) of a database in such a way that it can be re-imported into a newly installed database (from the same or a gither version of the system). This would not only allow to solve the upgrade challenge, but also to develop customizations for specific customer segments (customer fields, views, ...) and deploy them in a way that is independent of the master system.

For code changes the possibility to use the 'custom' directory solutions (copying the modified code in the 'custom' directory) works quite well, but a similar solution for the database is still missing (and I don't yet have a simple idea to offer how to achive it :()

[reeid]

I would go more with above... All above in weekly script chunks plus another script reverting last changes (database and files). Maybe there is a way to program scripts without much hassle?

[bpabiszczak]

1. The prolonged time between 2.0 and 2.1 updates is caused by bootstrap 3.3.4 upgrade. Upgrading bootstrap itself is not a problem, but upgrading libraries generated a vast amount of issues, a good example may be select2 library, to which we had to upgrade the whole system [in relation to bootstrap update], unfortunately this library has over 350 opened issues [ https://github.com/select2/select2/issues ?!?] . It turned out that we had to update many standard libraries, because their creators could not cope with it. Luckily, we gradually close the issues, the system is getting better, and we are not planning any more of such major changes affecting the system's stability in the future.
2. 2.2 version will be released relatively soon [not more than a month after releasing 2.1] and above all it will include fixes improving the system [improvement of the calendar view, optimization and better responsiveness] . We plan to publish Y updates [as in X.Y.Z] every month, and X updates once every six months. The Z versions appear every day and they are unstable.
3. Starting from 2.1 version we are planning to open a STABLE version, where the reported critical issues will be fixed and we will release manual patches for them. However, like I said before, it will concern critical errors that prevent the users from using the system.
4. In case of the Z versions we will not be releasing patches because there is plenty of these changes and creating packages would be too laborious for us, therefore we generate packages when we close the version. Currently, there is a working version of the migrating package (from version 2.0 to version 2.1) and it can be found here: https://github.com/YetiForceCompany/UpdatePackages/tree/developer/YetiForce%20CRM%202.x.x/2.0.0RC_to_2.1.0RC (at the moment we are testing it internally). Sometimes we generate these working versions, such as this one once a week [when we need a new functionality in our clients' or our own system]
5. You are making your life harder if you currently introduce custom changes and modify system files because

5.1. Most of the changes can be overwritten without modifying the system files, for instance: https://yetiforce.com/en/developer-documentation/engine/177-loader-and-autoloader-of-files.html

5.2. If you can not change the system in order not to modify the system files, why will you not add a functionality like that ? In this way you can solve the problem comprehensively. Remember, it is open source, we always willingly introduce good ideas and ready solutions.

[zebrahosting]

Thanks for the write up Blazej. Let me say it clear again. I did not write this to complain about the delays, far from that. I think it is good to test properly before releases, that always takes (more)time. Glad to see the manual (security)patches after version 2.1. That might help with those issues.

When I am talking about customizing, I am simply talking about making extra fields in for example contact details. Not in the system files. But still they are a lot of work I can't easily be transferred to a fresh installed version. So upgrading is the only way for that. If I use the 2.0RC to 2.1RC now I will miss the bug fixing in the next few days and I am afraid I can't update again to the same version.

One of the ideas above is to find a way to transfer custom made fields and/or modules for example out of the CRM and import them in a fresh installed version. Now we can only export the data but have to make sure all custom fields are made before the import again. Just good thoughts for the future.

[waw555]

Is user-defined fields are affected when you upgrade?

For them, in fact allocated a special table prefix or cf has something changed?

There CMS Joomla which the developers website, it is updated from zip archive or a socket, and there is sort of a database is also changing, but the user data remains, it may look as embodied?

[zebrahosting]

No of course I can upgrade/update without issues, but like many here I run versions from v1.0.0 and sometimes it feels more safe after all the big changes, to start with a clean setup and we dream of export/import of customized items to avoid all the extra work. You are right, this has nothing to do with the normal update procedure, but yes we do get carried away in dreams sometimes :-)

Bottomline of this discussion is about a proper way to have quick fixes in between point updates. Relevant because the original plan of monthly updates does not match the reality and sometimes security issues or major bugs should be fixed in between the bigger updates. Since Blazej has acknowledge to at least make security patches and hopes to bump up the speed of updates, it might work. Lets see in the next few month and in the mean time dream up of improvements and suggestion.

[bpabiszczak]

Unfortunately, I don't really understand your problem. Everything you change in the system from the configuration panel [you create fields, views, widgets, workflows] stays the same after upgrading. If, for some reason, we change something [we delete a field, add a view, change the order] it will only be because we think it's right and that's how it should be in the default system version, although you don't have to accept these changes.

If you update your CRM to a new version and we for example delete some field, you should be able to notice it while testing the system during the test upgrade. If you find all the elements you don't agree with, you change them in the update package and in the end your system will be what you expect it to be. We use exactly the same update packages for our clients, if the package deletes a field the client uses, then we remove this change for that particular client.

Worse if you don't have enough programming knowledge to change that or if you don't have an external company that could help you with it.

[florek41]

Błażej, you have right, but... :) @zebrahosting talking about export all configuration options from admin panel. It will be useful in few cases, ex:

• when you have test/dev version - you can configure it, export all options and import to production without migrate database.
• when you trying to change something for tests in production, with users etc - you can save all options in safe place and, in case of emergency, restore it without need to do a lot of work with manual changes.

I know that wordpress is different software, but there you can export every option to XML and later restore it.

[bpabiszczak]

Adding a mechanism that would allow exporting the entire configuration to XML for example might mean about a dozen weeks of work [if not more]. At the moment we are not planning to introduce that, and since the process is very complicated [it generates tons of problems] I don't think we'll add it within the next year.

Another thing – it's worth remembering that in comparison to quite easy to configure Wordpress, YetiForce is full of dependencies [a good example of this are permissions, scattered among several interdependent modules, and all that is stored in tens of tables], unfortunately it's designed like this, and changing it for now makes no sense because the benefits resulting from import/export configuration are inadequate to the time we would have to devote to achieving that.

[florek41]

It's clear to me. I understand and agree with Błażej. On the other hand - is there any other way to move all config (database tables etc.) ?

[bpabiszczak]

The configuration is around 200 tables, I don't think anybody would agree to separate them. We're planning to do it soon, in our free time... At the moment i have no good news for you.

Do you have any other questions regarding this topic or can I close?

[florek41]

It's clear, feel free to close.