Closed GoogleCodeExporter closed 9 years ago
Tried a complete export today, on my 2.6GHz-4GB, Ubuntu Precise system. I
started it from the commandline using
$ php5 -f ./download_xml.php includeimages=true includevideos=true | tee
~/CandyDollDB_2012-07-01.xml
It ran for one hour, then got killed by PHP:
PHP Fatal error: Maximum execution time of 3600 seconds exceeded in
/var/www/candydolldb/class/class.tag2all.php on line 29
PHP Stack trace:
PHP 1. {main}() /var/www/candydolldb/download_xml.php:0
PHP 2. XmlOutputModel() /var/www/candydolldb/download_xml.php:193
PHP 3. Tag2All::FilterTag2Alls() /var/www/candydolldb/download_xml.php:101
PHP 4. Tag2All->getModelID() /var/www/candydolldb/class/class.tag2all.php:190
Since watching the scripts' output, I am convinced the most time is spent in
Tag-filtering and Date-filtering. I would like to see a more efficient filter
mechanism. If that means calling an extra DB-query, so be it.
Original comment by fwp...@gmail.com
on 1 Jul 2012 at 8:34
Attached is a screenshot proving my point. It shows the vast majority of
processing (88%) takes place in Tag2All::FilterTag2Alls.
This analysis was created with the XDebug profiling output of a full XML-export
of VikaZ and loading this into KCacheGrind.
Original comment by fwp...@gmail.com
on 1 Jul 2012 at 6:11
Attachments:
Yeah 1 hour does seem a little long
Original comment by mranimos...@gmail.com
on 2 Jul 2012 at 8:52
Tweaked the filtering of tags in r528. Instead of filtering ALL tag2alls each
and every time, it now filters the big pile for all a model's tags, with the
other parameters set to FALSE. It then filters this array during set-, image-
and video-tagging.
It seems a LOT more efficient. The full export run of ALL models now takes less
than 8 minutes. Issue resolved, I would say.
Original comment by fwp...@gmail.com
on 3 Jul 2012 at 4:03
Original issue reported on code.google.com by
fwp...@gmail.com
on 1 Jul 2012 at 8:27