Slow response when adding multiple (thousands) files.

[Vogelhaus]

Hi,

I am evaluating the performance of PhileCMS. Overall a great "simple" CMS. Unfortunately I encountered the problem that if you add a lot of content pages ( I am talking about 100 / 1000) it slows all down. Requests take 10s for 10 000 pages.

Why is it necessary for PhileCMS to loop over all the content pages on each request?

[NeoBlack]

I agree, that we can maybe improve the handling, but the idea for PhileCMS was never to have big sites. It was designed for small websites. PhileCMS works without any kind of database, if you have any suggestion how to improve, I would like hear your ideas.

[kicken]

Why is it necessary for PhileCMS to loop over all the content pages on each request?

I believe the culprit is the {{pages}} variable for the template. In order to create it Phile needs to do a recursive search for all available content pages.

If you do not need this variable (ie, navigation is handled some other way) then disabling it should speed things up.

Perhaps a setting to skip populating this variable, or some way to make it only populate if used would be beneficial?

[james2doyle]

Can we cache the {{ pages }} call also? On Sun, Jun 28, 2015 at 1:01 PM kicken notifications@github.com wrote:

Why is it necessary for PhileCMS to loop over all the content pages on each request?

I believe the culprit is the {{pages}} variable for the template. In order to create it Phile needs to do a recursive search for all available content pages.

If you do not need this variable (ie, navigation is handled some other way) then disabling it should speed things up.

Perhaps a setting to skip populating this variable, or some way to make it only populate if used would be beneficial?

— Reply to this email directly or view it on GitHub https://github.com/PhileCMS/Phile/issues/256#issuecomment-116321681.

[Vogelhaus]

It would be also helpful if $pages was more specific. Maybe I just want the pages of a specific content folder. like $pages["documentation"]

Ok. I analyzed the code: Twig.php in line 162: $repository->findAll() refers to Page.php (Repository/Page.php) findAll Method

Temporary fix for Line 162: set array element pages to null.

Works like a rocket now.

[Schlaefer]

It would be also helpful if $pages was more specific. Maybe I just want the pages of a specific content folder. like $pages["documentation"]

You can test for page meta-data if you only want to act on a subset. E.g. with twig:

{% for page in pages %}
    {% if page.folder == 'documentation' %}
        // do if documentation page …

This default behavior should serve most people well out of the box, but depends on pages knowing about all pages. If the default pages variable doesn't fit your needs its easy to create your own in a custom plugin. Use the findAll method you already found to grab what you want and populate a template variable with it.

[Vogelhaus]

This default behavior should serve most people well out of the box, but depends on pages knowing about all pages. If the default pages variable doesn't fit your needs its easy to create your own in a custom plugin. Use the findAll method you already found to grab what you want and populate a template variable with it.

And this is I think the reason why it could be inefficient for some use cases. I don't say its totally "bad" and I am sure it suits smaller web pages very well. But in case you have a deep directory structure it will have to do many I/O cycles to retrieve all the pages in these folders.

So maybe even smaller sites with around 100 pages in 25 different folders could be potentially affected. I will try that out.

[Schlaefer]

I benchmarked recently¹ and up to 100 pages should be fine. After that it becomes a question of server, traffic, cache engine etc. A few hundred pages may be doable. My educated guess is that at 1000+ pages code tweaking is always required (at least as of now).

I will try that out.

:+1: The more data points the better.

¹For this this cache plugin (and some other code). If you're performance sensitive, you could give it a try.

[Schlaefer]

fixed in #257