search

modxcms / revolution

MODX Revolution - Content Management Framework
https://modx.com/
GNU General Public License v2.0
1.36k stars 529 forks source link

Resources Tree TOO SLOW, and it reloads to many times. #2510

Closed maxpiepenbrink closed 10 years ago

maxpiepenbrink commented 14 years ago

mido created Redmine issue ID 2510

The resources tree loads too slow. Each time we save or edit a document, or when we click any link, tree must be reloaded. A loading screen appears over the tree and it generates the tree child by child, taking several seconds each time we load a new page. A video example can be found here: http://www.youtube.com/watch?v=lO9SNp0HbRg

And two forum posts:

http://modxcms.com/forums/index.php/topic,53535.msg310548.html http://modxcms.com/forums/index.php/topic,53996.msg312172.html

We have less than 100 pages and it is slow for us, imagine a site with thousands of pages!

Can the tree be cached? TYPO3 do that. I think we should find a solution because it is very frustating to wait for the tree to finish loading to do something :(

wshawn commented 14 years ago

wshawn submitted:

Have you upgraded? How much memory do you have in your php.ini file? What OS are you using? As all of us are not experience this issue, it would be beneficial for you to provide us with information concerning your situation. Update this post with your current environment.

The manager is blindingly fast now and has been for the last few versions.

If you are on a cheap host, with an overloaded system, database, or minimalistic php memory settings things will get slow -- no matter what you run.

wshawn commented 14 years ago

wshawn submitted:

After watching your video, you are either running a beta or an early release version. Please update your Revolution installation to 2.0.4

maxpiepenbrink commented 14 years ago

mido submitted:

I thought it was a general problem, not only mine. Now we have 1024M memory limit. memory_limit 1024M 1024M

PHPINFO here: http://aeromedica.netfactory.es/manager/info.php

MODx version for video 2.0.0 or 2.0.2 ( i dont remember ). Now we have 2.0.4 and still the same.

It happens in all browsers and in all SO's

Server is a shared hosting on Dathorn.

maxpiepenbrink commented 14 years ago

mido submitted:

...

maxpiepenbrink commented 14 years ago

mido submitted:

I attach a system info screenshot ( it cannot be pasted! ) and a new youtube video... http://www.youtube.com/watch?v=VDhcC1WktBI

maxpiepenbrink commented 13 years ago

mido submitted:

Same problem reported here: http://praegnanz.de/weblog/five-things-i-don-t-like-about-modx-revolution

_1. Backend Performance

One of the things I liked most about Textpattern and the traditional MODx (called “Evolution”) is their super-fast backend. Sadly this aspect does not get a lot of attention in today’s CMS conversation. Still, it’s oftentimes critical to get your shit done very quickly, be it content editing or templating.

In MODx Revolution, however, the backend is made with ExtJS, which is a framework for building complex user interfaces with HTML, CSS and JavaScript. It certainly does its job in a super-cool object-oriented and MVC-driven way, I don’t know. Obviously, ExtJS wraps twenty div containers around each form element and embeds several tons of JavaScript to make the UX shiny and consistent across all platforms. Unfortunately, it’s slow as hell! Buttons don’t respond when clicked at the wrong pixel. The always-present page tree ajax-reloads after almost every operation, even if it had nothing to do with the page tree. Changing from one view to another makes you wonder why they didn’t just used frames like in MODx Evolution.

This surely isn’t about connection speed (it happens even locally), nor a specific browser (Chrome’s V8 engine should be fast enough). The current responsiveness of the backend might be okay for casual content editing, but while hacking together dozens of templates, snippets and system settings, it just steals way too much time.

(Almost ironic: The most inconvenient task is configuring and simplifying the backend for editors. You have to fill out a cryptic popup-form for every single change, which can be a nightmare.)_

modxbot commented 13 years ago

rethrash submitted:

Has this been re-tested with the latest 2.0.7 release and the compression turned on?

maxpiepenbrink commented 13 years ago

mido submitted:

I've upgraded to 2.0.7 , it's only a little bit faster, but still slow for me :( I've enabled some cache options in MODx. What do you mean by compression? Some MODx feature o serverside compression?

maxpiepenbrink commented 13 years ago

mido submitted:

I have migrated one website to a windows platform and the problem persists. The new server is:

Each reload of the tree is too slow!

modxbot commented 13 years ago

heavensbest submitted:

I am on 2.1 pl and it seems slower than 2.1 rc-4.

The Element tree is good. But the Resource tree sometimes loads, sometimes doesn't, sometimes halfway. It seems to slow down the farther into the Resource tree I get. 3, 4, 5 levels in.

I'm trying to get my system info, but have to keep hitting refresh. Because it is just not loading.

I think: Linux php 5.? Using Chrome

Been waiting for a minute or so and still not getting the System Info to show in Modx. It maybe my server, But it Definitely slows down the farther into the Resource tree I go. But with Elements, it doesn't matter how far into the tree I go, it seems to be just as good.

modxbot commented 13 years ago

heavensbest submitted:

I do think though, that the more that you use modx, the faster it gets. Because the browser caches more stuff.

I do appreciate modx Revo A LOT. It is just sometimes slow.

modxbot commented 13 years ago

hsteel submitted:

"This surely isn’t about connection speed (it happens even locally), nor a specific browser (Chrome’s V8 engine should be fast enough). The current responsiveness of the backend might be okay for casual content editing, but while hacking together dozens of templates, snippets and system settings, it just steals way too much time."

I have to throw my hat into this ring. I love modx Revo but the slowness of the tree and the manager in general is an issue and doesn't appear to be improving much (I admit it was extremely slow in the early versions and is a lot faster now but it's still not exactly fast - Michael's comment above pretty much summarises our studio's exact issues).

Also because Revo is so heavily dependant on caching, the cache often needs to be cleared before re-testing a site and this of course slows the manager down again. We host all our sites on pricy high-quality hosting specifically to get modx to run as fast as possible in a live environment and even that doesn't help much.

As I said, I love modx, I want to try to help improve it, and I'm one of (I'm sure) millions of users very very grateful to you guys for the amazing work you do :) This particular issue is just an ongoing drama for our entire development team and our clients and it would be great to see it addressed. If there's any additional info I can give you or testing we can do to help please let me know.

Thanks Harmony

maxpiepenbrink commented 13 years ago

mido submitted:

The problem is not cache or ExtJS itself, but the way the tree is loading. I posted this in forums:

For example, we have a small/medium website with 6 levels of content. Each time I do something (save, edit... ) the tree has to load again. If i'm in the last level and "save" a document, MODX reload the entire page and launches 15 ajax requests, 7 of them related to the tree to load each level and it takes several seconds to finish. Can it be handled in a best way? Why have you do you force a tree reaload each time? It's frustating to wait for the tree to reload when you have to make a lot of changes

There are a lot of ways for loading the tree. ExtJS has an option to load it at once. It could be loaded serverside too. It would be nice to have an option in system preferences to choose the way we want to load the tree.

Some time ago I read that loading the tree only in one connection could cause performance problems. But performance for who? For large sites I suppose. Small sites can work fine if we load the tree serverside or in only one ajax connection.

maxpiepenbrink commented 12 years ago

mido submitted:

Tested in MODX 2.2 but the problem persists. The tree continue being loaded async and it takes a few seconds to be fully expanded. I hope MODX could find a solution, so i'm not the only one with this problem

maxpiepenbrink commented 12 years ago

mido submitted:

Problem persists in MODX 2.2.0-pl! :(

ndh611 commented 12 years ago

ndh611 submitted:

Why no-one seems to care about this defect?

I love MODx Evo (still using them) but it seems like no-one is writing any plugins for them anymore. MODx Revo has great plugins like Gallery, Articles and the ONLY THING THAT STOPS me from using it for my client is the SLOW AS HELL resource tree.

Come on, on my local machine, it usually takes 15 seconds to simply open a page to edit. In Evo, it was instantly. The resource tree is reloaded every time, which is really inconvenient...

MarkH commented 12 years ago

markh submitted:

I'd like to reiterate that this isn't a problem everyone is having, so perhaps we need more debugging and testing on individual servers that are having issues.

I just ran a quick test on my localhost with a resource tree with about 3500 resources (about 8-20 children on average) and expanding everything did crash Firefox, but it did that within 15 seconds. Each level (when manually opening) feels fast (.3-.5 seconds, maybe?) varying from 5 to 40 resources being loaded per level. The production site feels a tad slower, I'd say an average of .5-.8s per level with many resources :)

It is possible to pre-load children with ExtJS, but you would be loading children you may not even use that are taking more server-side processing time.

MarkH commented 12 years ago

markh submitted:

I have to add that the same site was insanely slow on another server running MySQL 5.0.51a, which is not supported for MODX and many other advanced php apps. After moving to my own server those problems were gone.

If you were to preload children through one request that may, like was said, be fine on smaller sites, but on larger sites it will render the site tree pretty much useless as loading times for the ajax request will grow exponentially... one evil against the other.

MarkH commented 12 years ago

markh submitted:

Had a stab at incorporating this based on the stored state of the tree (which is what makes it reload certain nodes), and pushed it to my clone: https://github.com/Mark-H/revolution/tree/refactor-preloadtree (MODX 2.2)

Getting the inital children based on the state works, but by passing in children from the first request it seems to have broken the regular behavior of sending out an AJAX request for non-loaded children, instead it just gets rid of the expand (>) icon in front of the node...

MarkH commented 12 years ago

markh submitted:

Managed to get that fixed too..

Sent a pull request: https://github.com/modxcms/revolution/pull/271

It can probably use some improvements still, but it is a start :)

maxpiepenbrink commented 12 years ago

mido submitted:

Each level (when manually opening) feels fast (.3-.5 seconds, maybe?) varying from 5 to 40 resources being loaded per level. The production site feels a tad slower, I'd say an average of .5-.8s per level with many resources :)

The problem is not the number of childs per level, but the number of levels. The tree will start loading after the full page load ( some seconds for page load ), and then, If you have for example a tree of 5 levels depth, it will take 2,5 seconds more ( .5 * 5 levels ) to load the full tree.

If you were to preload children through one request that may, like was said, be fine on smaller sites, but on larger sites it will render the site tree pretty much useless as loading times for the ajax request will grow exponentially... one evil against the other.

It was commented before that and easy way to fix is to have a system setting to choose how you want to load the tree: preloaded or dynamic ( as now ). I saw you sugested something simmilar in your pull request:

I'm thinking it may be convenient to add a system setting to toggle the preloading behavior. It may be less desirable to preload everything that was opened when having a huge amount of resources.

Really thanks for taking your time to fix this!!!!! :)

MarkH commented 12 years ago

markh submitted:

Just to add.. that pull request only preloads children that have been opened (the tree's state stored in session), not the entire tree.

The trouble is you have to choose between two evils... either many small requests for each level, or one request that takes longer depending on how many resources you have open. I modified the processor to load children of the current node it is processing (so it is processing a resource, it sees you had it opened in the tree state and it fetches the children resources). Basically it's still doing per level, except it does it all in one request.

Ideally, it would collect the resource IDs that should be shown, collect all their information in one request/from cache, and then map it all into the tree. That would probably double the performance or so with many different levels open, but means the entire processor has to be refactored and I simply didn't/don't have the time for that.

maxpiepenbrink commented 12 years ago

mido submitted:

I couldn't test your mod because problems in my testing server but I'll try to take a look again this week. You have done a lot Mark! This problem has been frustrating for me and other MODX users and you have been the guy that took some of your time to fix it, so, THANKS. MODX has not only big things to do, but to fix also a lot of small things that can frustrate users ( slow tree load, save button not changing status some times, etc. ). Really thanks for taking some of your time to fix it!! :)

maxpiepenbrink commented 12 years ago

mido submitted:

Hi Mark, Has this problem been fixed in 2.2.1 or you are still working on it? Thanks again for taking care of this! :)

OptimusCrime commented 10 years ago

This issue is fixed.

exside commented 10 years ago

Is it?? Can't see the PR from MarkH merged somewhere? And as of 2.2.14pl the tree is still slow as hell and continuously opening "unused" containers on each reload, even if I close them before a reload...

OptimusCrime commented 10 years ago

@exside: Weeell, it does not have anything to do with the original issue. The tree is as slow as it has to be as far as I know. A lot happened with 2.2 in terms of speed, I don't think there is much more to gain without refracting the entire manger.

Also, if you see any incidents of unused containers opening by themselves, I'd suggest opening a new ticket with the case clearly defined. This ticket is way too broad.