clearCompiledTemplates is inefficient

[GoogleCodeExporter]

I've been profiling a script in which clearCompiledTemplate is called 4 times 
(it's one Smarty (3.1.6) instance with 4 template resources). My compiled 
templates folder contains lots of files and folders. This results in 10,254 
calls to Smarty_Internal_Utility::clearCompiledTemplate, and in turn, 37,930 
calls to rmdir that fail because the dir is not empty. I am using a custom 
error handler, but it checks for error suppression and so doesn't log anything, 
but still, this seems like it's doing something horribly wrong to be calling 
this so many times for so few resources!

It appears to clear a compiled template by traversing the entire templates_c 
tree (or subtree if there's a compile_id), checking if each item (about 37,000 
of them in my case) has anything to do with the current template, and 
attempting to delete *every* folder in the tree, relying on the CHILD_FIRST 
ordering and error suppression to not delete things that are not to do with the 
current template, i.e. the time taken to delete one template is proportional to 
how many compiled templates there are, and it's likely to have unnecessarily 
large memory overhead - in my case it's traversing 37,000 items in order to 
delete just 4 of them. This seems crazily inefficient. Is there not some 
simpler mapping from a template resource to which files it relates to so that 
files can be deleted in O(1) time?

Or have I misunderstood the code?

I cleared my compiled templates folder and the issue went away, but that's only 
a temporary fix.

Original issue reported on code.google.com by marcus.b...@gmail.com on 26 Feb 2014 at 11:42

[GoogleCodeExporter]

1. If you call $smarty->clearCompiledTemplate() it calls 
Smarty_Internal_Utility::clearCompiledTemplate() just once.
So you can't see 10254 calls of 
Smarty_Internal_Utility::clearCompiledTemplate() for 4 times calling 
$smarty->clearCompiledTemplate(). There must be something wrong in your 
observations.

2. You say you had about 37000 files in your template_c folder. Why do you have 
so many files? It looks like that you have used a lot of different 
compile_id's, why?
For what do you use compile_id's? Could it be a misunderstanding of it's usage? 
It is very unusual to have so many files.

Original comment by Uwe.Tews@googlemail.com on 28 Feb 2014 at 6:37

[GoogleCodeExporter]

I only have a single compile_id for this particular collection (in all I have 
three compile IDs but only one is used in the part I'm looking at). They are 
templates for email messages; they get used for a batch then not used again, 
and they build up over time. Inside the templates_c folder smarty automatically 
creates lots of nested folders with 2-character names - I've not checked if 
they all have something in them.

Having a large number of compiled templates shouldn't be a problem. But even if 
there were not that many I still don't understand why it traverses the entire 
tree to delete just a few files.

The numbers came from an xdebug profile; all the other numbers are correct, so 
I'm inclined to believe them.

Original comment by marcus.b...@gmail.com on 28 Feb 2014 at 7:00

[GoogleCodeExporter]

See point 1. in my previous message. If you call 
$smarty->clearCompiledTemplate() only 4 times you can't have 10254 calls of 
Smarty_Internal_Utility::clearCompiledTemplate(). You should recheck this.

So if I understand it right now you create a lot of template names when you run 
your batch which are not reused. This would be a use case where the structure 
of the template_c subfolders is not optimized for.

It may be better to use $smarty->use_sub_dirs = false; and call 
$smarty->clearCompiledTemplate(); (no parameter) at the end of your batch run 
to get rid of the compiled templates.

Smarty3.1 does use same sub-directory structure as Smarty2. 
There will be some optimization in the next major version 3.2.
Currently it does traverse through the folders as a call like 
$smarty->clearCompiledTemplate('foo.tpl');  must catch also entries with any 
compile_id.

Original comment by Uwe.Tews@googlemail.com on 28 Feb 2014 at 8:01