search

SocialEngine / phpv4-feature-requests

The purpose of this repository is to collect SocialEngine PHP public feature requests.
https://www.socialengine.com
1 stars 0 forks source link

Deleting old plugins #36

Open DonnaScriptTechs opened 7 years ago

DonnaScriptTechs commented 7 years ago

From @poeticjustice on February 13, 2017 14:24

Just wondered if any plans for SE to add a feature that will fully remove plugins and their database entries into MySQL

I was recently removing a plugin and followed the guide provided https://support.socialengine.com/php/customer/en/portal/articles/2121821-how-to-remove-delete-a-plugin-from-your-website?b_id=14386

But out of curiosity i thought ill reinstall the plugin, once installed i found the plugin was already activated with its licence key and all its settings even tho i deleted all entries it made in the my.sql file in the settings folder of the plugin, but because of the mentioned it did not add the queries to the database, instead it completely ignored all entries into the database, because there was entries in other tables for the plugin, which are not shown in the my.sql file, there was no referance anywhere to where these random files were hiding, but they were mainly all injected into the core mysql parts like core_settings core_activity_types etc... but as mentioned no file in the settings of the module had any referance to these new entries in the database.

because these hidden entries the plugin will not install at all now, lucky on my staging server, but would be great if there was any more information that could be updated to this help file, or even a feature that asks Do you want to delete entries in database also or just remove plugin

Copied from original issue: SocialEngine/phpv4-issues#555

DonnaScriptTechs commented 7 years ago

From @Khushboo016 on February 14, 2017 13:17

Hi @poeticjustice,

Regarding "its settings even tho i deleted all entries it made in the my.sql file in the settings": After reading this line, it seems to me that if you've deleted the entries from this file /application/modules/MODULE-NAME/settings/mysql file, but as we have clearly mentioned in the article that in order to delete the entries of the deleted plugins from the database of your site, it is required to undo (not delete the queries from the file) all those SQL queries in the database of your site. For example, if a Create Table query had been run, you will need to delete the table that was created. Likewise, if there was an Insert query, whichever rows were inserted will need to be removed from the database.

So, please make sure you undo the queries in the database of your site and in order to perform this step to undo the queries, you should have some database knowledge, otherwise you may delete some essential data from the database.

Thanks & Regards!

DonnaScriptTechs commented 7 years ago

From @gsf00001 on February 14, 2017 16:17

A agree - SE needs to include a removal tool/option. Disabling often does nothing (since many plugins still do all sorts of time-wasting checks even when disabled). I currently have >20 that I want to remove and one Dev quoted me .5hr/ea which is very reasonable. Another quoted about 18 or so hours to create a removal tool. Not sure if either of these does a complete job, but I was obviously leaning toward the tool since it could be used down the road. I've also experienced conflicts where although I disabled a Theme by one Dev, it still conflicted with an Enabled theme by another Dev.

The point is that if an app allows installing something, it should allow the removal of it (not just enable/disable - especially when disabling typically does practially nothing). This should not be a request - it should have existed many years ago.

I would add another considerations as well: Disable should do a complete job of making the Plugin "invisible" i.e. as if it was never installed.

DonnaScriptTechs commented 7 years ago

From @poeticjustice on February 14, 2017 18:17

Hi @Khushboo016

Slightly confused me here, you say "undo (not delete the queries from the file) all those SQL queries in the database" then shortly after you say "For example, if a Create Table query had been run, you will need to delete the table that was created. Likewise, if there was an Insert query, whichever rows were inserted will need to be removed from the database."

I am basically deleting tables created and inserts made into existing tables...

The part im getting at tho is, having undoing whats mentioned in the MySQL file of the plugin, im finding more entries still in the databases related to the plugins, such as the serial keys and the fact the plugin is enabled. These seem to be entries not made by the plugin, but what im guessing social engine is adding or making...

Example: no where in the MySQL file does it mention create table or insert into engine4_core_settings But in engine4_core_settings there is loads of entries referencing the module such as serial key etc and others

also in engine4_core_menuitems there is loads of rows relating to the plugin, but in the MySQL file it only mentions adding one, none of the other 15 created.

also engine4_core_mailtemplates has an entry thats not mentioned in MySQL file

entries to engine4_core_content but nothing mentioned in MySQL which seems to ref the widget

engine4_activity_notificationtypes has entries also

DonnaScriptTechs commented 7 years ago

From @GabryNet on February 15, 2017 9:16

I agree that we need some improvements about the complete removing of plugins ... folders we can remove them also manually but the big problem is with the database ... removing plugins leave a big mess in database and also errors on the website ... i will give an example with a plugin that i used for advanced notifications ... after removing this plugin, instead of default notifications taking over the job i had errors on opening the notifications saying that i have to contact the administrator and a number of error ... i had to manually delete from database the entries for this plugin to make notifications to work again ...and this was a simple plugin so i could find the entries easy ...imagine what's happening if you disable 3-4 "big" plugins, like i said it will be a mess and headache to clean after.

DonnaScriptTechs commented 7 years ago

From @SpiritWolfie on February 18, 2017 6:31

I fully agree too, the ability to use a removal option or tool would resolve allot of issues

This is a issue I have as well! I can confirm how much of a headach this is.

DonnaScriptTechs commented 7 years ago

From @poeticjustice on February 18, 2017 18:11

Good to see others agree, i just feel Social Engine need to update the info on this page also https://support.socialengine.com/php/customer/en/portal/articles/2121821-how-to-remove-delete-a-plugin-from-your-website?b_id=14386

because they say

"When you uninstall a plugin, it may leave behind some database rows or tables, leaving you with a cluttered database. You can remove these by opening the /application/modules/MODULE-NAME/settings/mysql file and undoing all the queries written in that file that pertain to the plugin you have uninstalled."

But as i found Social Engine itself is making MySQL entries for these plugins in random areas not mentioned in above file...... SE should have noted serial keys and the menus and other areas made likes widget pages etc could still have mysql enteries for the plugin

DonnaScriptTechs commented 7 years ago

From @Elshara on February 22, 2017 9:26

I agree with this concept. One method you could use for doing this is search the database for recent entries. It's a long shot but if you could filter it some how you might have a chance here at finding a needle in a hay stack. The problem is you're never alerted as to what database queries are actually posted when a module is installed. If so, you'd know where to go to remove it manually.

DonnaScriptTechs commented 7 years ago

From @Elshara on February 22, 2017 9:30

Also, the file method does nothing. If you don't have php my admin installed, you have a problem. All you need is a database monitor tool you can add to the database manager. Until social engine decides to take removing of packages seriously, and make sure not only modules, but member data is handled in their own isolated environments. Removing each environment if necessary to ensure complete package or data removal as it would not effect core product functionality.

DonnaScriptTechs commented 7 years ago

From @gsf00001 on February 23, 2017 21:29

Plus, as I've found out from some 3rd-Party Devs, they often 'do more stuff' with integration of their other Plugins, thus a simple remove/reverse/undo type of procedure may not undo everything - so this remove/uninstall feature may require 3rd-Party Devs to include the proper procedures/code to completely 'undo' the entire install process completely. This seems to be a little more complex than we might think, but it obviously can be done (and should be). With over 20 Plugins I'd like to remove -this is a big deal. As I mentioned earlier I could pay close to $300-500ish to have a Dev manually do this or create an app that will do it - neither of which will completely 'undo' everything as well as all Plugin settings that were made during installation.

DonnaScriptTechs commented 7 years ago

From @Elshara on February 23, 2017 23:37

Nothing is easy when it comes to social engine and complex database tables. Dolphin Pro actually gives you the option to back up your database before you started adding stuff to take care of problems like this.

On 23/02/2017, gsf00001 notifications@github.com wrote:

Plus, as I've found out from some 3rd-Party Devs, they often 'do more stuff' with integration of their other Plugins, thus a simple remove/reverse/undo type of procedure may not undo everything - so this remove/uninstall feature may require 3rd-Party Devs to include the proper procedures/code to completely 'undo' the entire install process completely. This seems to be a little more complex than we might think, but it obviously can be done (and should be). With over 20 Plugins I'd like to remove -this is a big deal. As I mentioned earlier I could pay close to $300-500ish to have a Dev manually do this or create an app that will do it - neither of which will completely 'undo' everything as well as all Plugin settings that were made during installation.

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/SocialEngine/phpv4-issues/issues/555#issuecomment-282127063

DonnaScriptTechs commented 7 years ago

From @GabryNet on February 24, 2017 6:3

Hi @Elshara , regarding the backup i think we all do this thru Cpanel or similar software , but i will give you an example when also the backup does not help ... if i buy a plugin for advanced members , i will make a backup before installing , then i install it and use it 3 months , during this time i also install other plugins ... after 3 months appears another advanced members plugin from another company that i like it more and i want to use it , so i have to disable-delete the old plugin and put the new one .. here the problems are starting as i can't use the old backup because during 3 months the website had a lot of new posts, changes, plugins etc . and i don't want to lose them ... that's why we need a good way to clean the database after deleting.

DonnaScriptTechs commented 7 years ago

From @Elshara on February 24, 2017 12:32

Seems like a good database monitoring tool would be most useful here.

On 23/02/2017, GabryNet notifications@github.com wrote:

Hi @Elshara , regarding the backup i think we all do this thru Cpanel or similar software , but i will give you an example when also the backup does not help ... if i buy a plugin for advanced members , i will make a backup before installing , then i install it and use it 3 months , during this time i also install other plugins ... after 3 months appears another advanced members plugin from another company that i like it more and i want to use it , so i have to disable-delete the old plugin and put the new one .. here the problems are starting as i can't use the old backup because during 3 months the website had a lot of new posts, changes, plugins etc . and i don't want to lose them ... that's why we need a good way to clean the database after deleting.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/SocialEngine/phpv4-issues/issues/555#issuecomment-282212073

DonnaScriptTechs commented 7 years ago

From @Khushboo016 on March 2, 2017 8:10

Hi @poeticjustice, @gsf00001, @SpiritWolfie, @Elshara, We have added this suggestion at our end and hopefully, we would be able to make such a feature possible in future in SE PHP which would allow complete removal of plugins from SE PHP site along with the removal of their entries from the database.

Thanks & Regards!

DonnaScriptTechs commented 7 years ago

From @Elshara on March 2, 2017 23:45

Awesome news! Thank you so very much!

On 02/03/2017, Khushboo016 notifications@github.com wrote:

Hi @poeticjustice, @gsf00001, @SpiritWolfie, @Elshara, We have added this suggestion at our end and hopefully, we would be able to make such a feature possible in future in SE PHP which would allow complete removal of plugins from SE PHP site along with the removal of their entries from the database.

Thanks & Regards!

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/SocialEngine/phpv4-issues/issues/555#issuecomment-283585065

DonnaScriptTechs commented 7 years ago

From @gsf00001 on March 30, 2017 18:21

(if this should get moved elsewhere please do so - I don't want to hijack this post) Silly question (please bear with me for a moment ...)

Q1: Will removing Plugins actually speed things up considerably - i.e. has anyone performed actual tests? Q2: Should disabling Plugins actually speed things up at all?

Note that I'm not asking about any other benefits (saved storage space, etc.) - strictly performance. My current Page loads are around 7s (well, for the page to look loaded and be functional, then 1-2dz more sec to fully load). I'd like to reduce this.

RE: Q1 I ask this because I have 20+ Plugins to delete and don't want to pay to have this performed manually (or pay for 'delete Plugin' Plugin) if the performance increase is negligible. And since I obviously haven't done this I can't say if this will increase performance at all.

RE: Q2 This I have actually been able to test, and so far the performance increase seems to be negligible at best. Original Status:

Test w/Results (I'm just using Chrome's Dev Tools - nothing fancy - 1 User so obviously no load-testing happening here):

Soooo... all of that to say that disabling 91 Plugins and 3 Cores seemed to make no negligible difference in page load time.

Back to my original questions:

What I'm trying to determine is what makes the biggest performance difference from the below:

This will help me decide if the ability to remove plugins makes any difference performance-wise or if it's just 'neater' in the database and saves a little disk space. Thanks for your input.

DonnaScriptTechs commented 7 years ago

From @Elshara on March 31, 2017 4:25

I can tell you this from experience.

  1. Disabling plugins only makes a difference if the plugin files aren't being loaded throughout the site. You'll actually have to remove all the files of the plugin as well as all database queries associated with it to see the difference. Also, be sure to manually remove all widgets and columns from new and existing pages and their associated ID columns in the layout editor such a plugin was placed on. Only then will you see any performance tweaks applied as if it wasn't there. Disabling a plugin just removes it from the index query in the administration panel. IT does nothing else. You actually have to manually remove it to even notice it was ever gone.
  2. Because removing and disabling plugins go hand in hand, your performance of on site widgets per page takes the biggest hit. The reason why your performance is so slow, is to do with tasks running in the task scheduler as well as widgets loading on the page simultaneously, maximizing connections to the database MYSQL queries can handle all at once. If you find a plugin or module useful, keep it. Its existence won't prevent the inevitable, but it will if removed, and all widget references in layout editor as well as database, menu editor, files and so on, result in performance boost significantly. The question you need to ask yourself is. How much data am I loading per page request, and what can I do to minimize it based on widget queries to the database happening all at once? Whatever dependencies you have for the network to load first, must be loaded as a priority. Believe it or not, this takes the most time to do so. Then, if the widgets themselves are individually waiting for load times and queries to complete, this is what you are waiting to be filtered through the dependencies still waiting to be loaded. Everything is happening all at once, and sometimes, this is what causes the browser to be unresponsive, as the database reaches its maximum connection limit as the queries try to compete with each other one at a time while the request is ongoing. It actually causes lots of stress on the server, due to maximum connections set via MYSQL preferences. So if you are going to notice things load as per optimal performance, you're going to need to customize the time out queries and connections can contain through database adaptation. Prior to seeing real performance gain. I know this is not the answer you want to hear, but it's the only one I've got at the moment. I have no idea if social engine has any plans on improving such statistics, but this is just the tip of the ice burg here. From my limited understanding of the technical specifications of loading data at a glance. I've tested my theory out here and I estimate every extra column of data a database cell has to load on page request, requires a minimum of 200 - 400 milliseconds to complete, based on server ping time at active engagement with other database cells at the same time per every one cell it simultaneously is currently accessing at once. Times that by however long it takes to process the data through the Zend template engine, doubling the amount of connections to the database. filter it through php for every individual request simultaneously. Serve it through the index.tpl file. Load all the javascript, css styles and other external documents the same way. Output all the data in order through the combined effort of all the data that has been loaded into a dom output for the document. Create a temporary file using FtP requiring further database access to retrieve the stored information to do so. Set all the FTP commands for the file. Cache the newly created FTP file in a temporary caching engine. Wait for the caching engine process to run in task scheduler, assuming it won't time out due to maximum database connections. Process the file information to the server so it can remain in the cache for however long it is currently being accessed. Record such access point in yet another connection to the database. Connect the request of the browser ID to such a database connection using an overlapping database query overhead, requiring PHP and http additional network round trips to do so. Then display it in some temporary file of html design for the browser to read. Times all that by the principle of however many filters are required to handle each piece or syntax and character of code and information, and you have your answer. This happens every time a page is loaded.

On 30/03/2017, gsf00001 notifications@github.com wrote:

(if this should get moved elsewhere please do so - I don't want to hijack this post) Silly question (please bear with me for a moment ...)

Q1: Will removing Plugins actually speed things up considerably - i.e. has anyone performed actual tests? Q2: Should disabling Plugins actually speed things up at all?

Note that I'm not asking about any other benefits (saved storage space, etc.) - strictly performance. My current Page loads are around 7s (well, for the page to look loaded and be functional, then 1-2dz more sec to fully load). I'd like to reduce this.

RE: Q1 I ask this because I have 20+ Plugins to delete and don't want to pay to have this performed manually (or pay for 'delete Plugin' Plugin) if the performance increase is negligible. And since I obviously haven't done this I can't say if this will increase performance at all.

RE: Q2 This I have actually been able to test, and so far the performance increase seems to be negligible at best. Original Status:

  • approx 150 pkgs/Plugins/Themes installed, approx 130 enabled (plus some that can't be enabled/disabled so they're obviously enabled by default)
  • as mentioned above, typical page loads are approx 7+s

Test w/Results (I'm just using Chrome's Dev Tools - nothing fancy - 1 User so obviously no load-testing happening here):

  • disabled another 91 (from SEAO, SEM, SEE, Radcodes, HE, Webhive, SE)
  • disabled Cores for Radcodes, HE, Webhive (since I disabled all their Plugins)
  • Basically, all Plugins are disabled EXCEPT FOR some SE, SEAO, SES
  • Only Cores that are enabled are SE (obviously), SEAO (Plugins), SES (Theme)
  • Testing results of page loads: still approx 7+s; the only difference I noticed was when using the SE Default Theme instead of SES Theme the page loads seemed to register approx .5s faster but that could be for many reasons (such as my header/footer and other Theme specific content not loading).

Soooo... all of that to say that disabling 91 Plugins and 3 Cores seemed to make no negligible difference in page load time.

Back to my original questions:

  • Q1 (I have no idea and ask for your input - based on actual testing not guessing)
  • Q2 (my testing has revealed Disabling makes no realistic difference - but maybe for certain Plugins and/or Cores it might make a difference) I ask for your input based on your actual testing

What I'm trying to determine is what makes the biggest performance difference from the below:

  • Installed Plugins

  • Which Specific Plugins
  • Which Specific Cores

  • Enabled Plugins

This will help me decide if the ability to remove plugins makes any difference performance-wise or if it's just 'neater' in the database and saves a little disk space. Thanks for your input.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/SocialEngine/phpv4-issues/issues/555#issuecomment-290499811

DonnaScriptTechs commented 7 years ago

From @gsf00001 on April 2, 2017 18:34

@Elshara So, would you suggest (for testing purposes) that I create a new folder and install only the minimal # of SE and 3rd-Party Plugins (in my case about 40 or so) in the hopes that this should be much 'faster'? Thx.

DonnaScriptTechs commented 7 years ago

From @Elshara on April 3, 2017 3:30

Something like that. Just for testing purposes you know. Just see after which one is installed manually, how the site performs. Loading pages, adding posts.

On 02/04/2017, gsf00001 notifications@github.com wrote:

@Elshara So, would you suggest (for testing purposes) that I create a new folder and install only the minimal # of SE and 3rd-Party Plugins (in my case about 40 or so) in the hopes that this should be much 'faster'?

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/SocialEngine/phpv4-issues/issues/555#issuecomment-291005313