search

smallsong / minify

Automatically exported from code.google.com/p/minify
BSD 3-Clause "New" or "Revised" License
0 stars 0 forks source link

minify takes up to 5min to compress js scripts when mbstring.func_overload is activated #176

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
Minify version:
PHP version:    5.2.12 

What steps will reproduce the problem?
1.deactivating mbstring.func_overload
2.
3.

Expected output:

Actual output:
hte full result but never less than 3min. responsetime

Did any unit tests FAIL? (Please do not post the full list)
sorry, will going to test it.
Please provide any additional information below. If this has to do with URI
rewriting, please include your DOCUMENT_ROOT, the

Original issue reported on code.google.com by i...@medic-world.de on 6 May 2010 at 10:42

GoogleCodeExporter commented 9 years ago 
fix from issue 132 is done. based on that i have that effect

Original comment by i...@medic-world.de on 6 May 2010 at 10:45

GoogleCodeExporter commented 9 years ago 
Your summary and steps to reproduce say two different things. Is JSMin slower 
before 
or after the fix is applied? How much slower? How large is the input? Feel free 
to 
attach a zip of the input.

In most cases Minify can combine the js/css before minifying, but you should 
try 
disabling this:

http://code.google.com/p/minify/source/browse/tags/release_2.1.3/min/lib/
Minify.php#443
Alter line 443 of Minify.php so that the condition always fails:

if (0 && Minify_Source::haveNoMinifyPrefs(self::$_controller->sources)) {

Does this offer an improvement?

Original comment by mrclay....@gmail.com on 6 May 2010 at 11:24

GoogleCodeExporter commented 9 years ago 
"..Is JSMin slower before or after the fix is applied.." before this fix i got 
fast
but incomplete response js/css was cut off. any try result with different cuts.

After that fix the response are always complete but took several minutes.

(sorry) my minify version is 2.1.3

your fix : "if (0 && 
Minify_Source::haveNoMinifyPrefs(self::$_controller->sources))
{" speed it up but never less 15 seconds.

any idea? is it a matter of gzip?

Original comment by i...@medic-world.de on 6 May 2010 at 1:35

GoogleCodeExporter commented 9 years ago 
The patch only changes a couple strlen calls so I don't see why it's crawling. 
You'd
have to insert benchmarking code to figure this out I'm afraid.

Original comment by mrclay....@gmail.com on 6 May 2010 at 1:52

GoogleCodeExporter commented 9 years ago 
our request looks like this:

/min/?f=templates/bmgtest/javascript/jquery-1.3.2.min.js,templates/bmgtest/javas
cript/jquery.flash.js,templates/bmgtest/javascript/jquery.form.js,templates/bmgt
est/javascript/jquery.history.js,templates/bmgtest/javascript/jquery.autocomplet
e.v1.2.js,templates/bmgtest/javascript/jquery.dependClass.js,templates/bmgtest/j
avascript/jquery.slider-min.js,templates/bmgtest/javascript/jquery.json-2.2.min.
js,templates/bmgtest/javascript/search.v1.2.js,templates/bmgtest/javascript/jque
ry.scrollTo-1.4.2-min.js,templates/bmgtest/javascript/base64.js,templates/bmgtes
t/javascript/jquery.fancybox1.3.0.js,templates/bmgtest/javascript/jquery.mousewh
eel-3.0.2.js

i know that a lot :-) (already set the max script count to 20).

Original comment by i...@medic-world.de on 6 May 2010 at 1:53

GoogleCodeExporter commented 9 years ago 
"...have to insert benchmarking code to figure this out I'm afraid." i'm not, 
can you
give me a hint where i should set breakpoints? i will keep you informed

Original comment by i...@medic-world.de on 6 May 2010 at 1:56

GoogleCodeExporter commented 9 years ago 
Best place I can imagine is before and after the calls to mb_strlen. So, in your
patched JSMin.php, before/after this line:

$this->inputLength = $is_mb_strlen ? mb_strlen($this->input, 'ASCII') :
strlen($this->input);

Original comment by mrclay....@gmail.com on 6 May 2010 at 2:03

GoogleCodeExporter commented 9 years ago 
btw, sorry issue is only about js files. css works fine

Original comment by i...@medic-world.de on 6 May 2010 at 2:03

GoogleCodeExporter commented 9 years ago 
here the min_unit_test/test_all.php output

PASS: Minify : 304 response (1 of 1 tests run so far have passed)
PASS: Minify : cache, and minifier classes aren't loaded for 304s (2 of 2 tests 
run
so far have passed)
!FAIL: Minify : JS and Expires (1 of 3 tests run so far have failed)
PASS: Minify : Issue 73 (3 of 4 tests run so far have passed)
PASS: Minify : Issue 89 : bubbleCssImports (4 of 5 tests run so far have passed)
PASS: Minify : Issue 89 : detect invalid imports (5 of 6 tests run so far have 
passed)
PASS: Minify : Issue 89 : don't warn about valid imports (6 of 7 tests run so 
far
have passed)
PASS: Minify : CSS and Etag/Last-Modified (7 of 8 tests run so far have passed)
PASS: Minify_Build : single file path (8 of 9 tests run so far have passed)
PASS: Minify_Build : multiple file paths (9 of 10 tests run so far have passed)
PASS: Minify_Build : file path and a Minify_Source (10 of 11 tests run so far 
have
passed)
PASS: Minify_Build : uri() with no querystring (11 of 12 tests run so far have 
passed)
PASS: Minify_Build : uri() with existing querystring (12 of 13 tests run so far 
have
passed)
PASS: Minify_Cache_APC : store (13 of 14 tests run so far have passed)
PASS: Minify_Cache_APC : getSize (14 of 15 tests run so far have passed)
PASS: Minify_Cache_APC : isValid (15 of 16 tests run so far have passed)
PASS: Minify_Cache_APC : display (16 of 17 tests run so far have passed)
PASS: Minify_Cache_APC : fetch (17 of 18 tests run so far have passed)
NOTE: Minify_Cache_File : path is set to: '/tmp'.
PASS: Minify_Cache_File : store (18 of 19 tests run so far have passed)
PASS: Minify_Cache_File : getSize (19 of 20 tests run so far have passed)
PASS: Minify_Cache_File : isValid (20 of 21 tests run so far have passed)
PASS: Minify_Cache_File : display (21 of 22 tests run so far have passed)
PASS: Minify_Cache_File : fetch (22 of 23 tests run so far have passed)
PASS: Minify_Cache_File : store w/ lock (23 of 24 tests run so far have passed)
PASS: Minify_Cache_File : getSize (24 of 25 tests run so far have passed)
PASS: Minify_Cache_File : isValid (25 of 26 tests run so far have passed)
PASS: Minify_Cache_File : display w/ lock (26 of 27 tests run so far have 
passed)
PASS: Minify_Cache_File : fetch w/ lock (27 of 28 tests run so far have passed)
PASS: Minify_Cache_Memcache : store (28 of 29 tests run so far have passed)
PASS: Minify_Cache_Memcache : getSize (29 of 30 tests run so far have passed)
PASS: Minify_Cache_Memcache : isValid (30 of 31 tests run so far have passed)
PASS: Minify_Cache_Memcache : display (31 of 32 tests run so far have passed)
PASS: Minify_Cache_Memcache : fetch (32 of 33 tests run so far have passed)
PASS: Minify_CSS : subsilver (33 of 34 tests run so far have passed)
PASS: Minify_CSS : comments (34 of 35 tests run so far have passed)
PASS: Minify_CSS : unusual_strings (35 of 36 tests run so far have passed)
PASS: Minify_CSS : issue62 (36 of 37 tests run so far have passed)
PASS: Minify_CSS : selectors (37 of 38 tests run so far have passed)
PASS: Minify_CSS : paths_rewrite (38 of 39 tests run so far have passed)
PASS: Minify_CSS : hacks (39 of 40 tests run so far have passed)
PASS: Minify_CSS : vladmirated (40 of 41 tests run so far have passed)
PASS: Minify_CSS : paths_prepend (41 of 42 tests run so far have passed)
PASS: Minify_CSS : styles (42 of 43 tests run so far have passed)
PASS: Minify_CSS_UriRewriter (43 of 44 tests run so far have passed)
PASS: Minify_CSS_UriRewriter : Issue 99 (44 of 45 tests run so far have passed)
PASS: Minify_CommentPreserver (45 of 46 tests run so far have passed)
PASS: Minify_CommentPreserver (46 of 47 tests run so far have passed)
PASS: Minify_CommentPreserver (47 of 48 tests run so far have passed)
PASS: Minify_CommentPreserver (48 of 49 tests run so far have passed)
<br />
Strict Standards:  mb_strpos() [<a
href='function.mb-strpos'>function.mb-strpos</a>]: It is not safe to rely on the
system's timezone settings. Please use the date.timezone setting, the TZ 
environment
variable or the date_default_timezone_set() function. In case you used any of 
those
methods and you are still getting this warning, you most likely misspelled the
timezone identifier. We selected 'Europe/Berlin' for 'CEST/2.0/DST' instead in
/home/www/httpd/xtcReuter-Dev/htdocs/min/lib/JSMin.php on line 113<br />
<br />
Warning:  mb_strpos() [<a href='function.mb-strpos'>function.mb-strpos</a>]:
Empty delimiter in /home/www/httpd/xtcReuter-Dev/htdocs/min/lib/JSMin.php on
line 113<br />
PASS: Minify_HTML (49 of 50 tests run so far have passed)
<br />
Strict Standards:  mb_strpos() [<a
href='function.mb-strpos'>function.mb-strpos</a>]: It is not safe to rely on the
system's timezone settings. Please use the date.timezone setting, the TZ 
environment
variable or the date_default_timezone_set() function. In case you used any of 
those
methods and you are still getting this warning, you most likely misspelled the
timezone identifier. We selected 'Europe/Berlin' for 'CEST/2.0/DST' instead in
/home/www/httpd/xtcReuter-Dev/htdocs/min/lib/JSMin.php on line 113<br />
<br />
Warning:  mb_strpos() [<a href='function.mb-strpos'>function.mb-strpos</a>]:
Empty delimiter in /home/www/httpd/xtcReuter-Dev/htdocs/min/lib/JSMin.php on
line 113<br />
<br />
Strict Standards:  mb_strpos() [<a
href='function.mb-strpos'>function.mb-strpos</a>]: It is not safe to rely on the
system's timezone settings. Please use the date.timezone setting, the TZ 
environment
variable or the date_default_timezone_set() function. In case you used any of 
those
methods and you are still getting this warning, you most likely misspelled the
timezone identifier. We selected 'Europe/Berlin' for 'CEST/2.0/DST' instead in
/home/www/httpd/xtcReuter-Dev/htdocs/min/lib/JSMin.php on line 113<br />
<br />
Warning:  mb_strpos() [<a href='function.mb-strpos'>function.mb-strpos</a>]:
Empty delimiter in /home/www/httpd/xtcReuter-Dev/htdocs/min/lib/JSMin.php on
line 113<br />
PASS: Minify_HTML (50 of 51 tests run so far have passed)
PASS: ImportProcessor (51 of 52 tests run so far have passed)
PASS: ImportProcessor : included right files in right order (52 of 53 tests run 
so
far have passed)
!FAIL: Minify_Lines (2 of 54 tests run so far have failed)
PASS: HTTP_Encoder : recognize "x-gzip" as gzip (53 of 55 tests run so far have 
passed)
PASS: HTTP_Encoder : gzip w/ non-zero q (54 of 56 tests run so far have passed)
PASS: HTTP_Encoder : gzip w/ zero q (55 of 57 tests run so far have passed)
PASS: HTTP_Encoder : IE6 w/o "enhanced security" (56 of 58 tests run so far 
have passed)
PASS: HTTP_Encoder : IE6 w/ "enhanced security" (57 of 59 tests run so far have 
passed)
PASS: HTTP_Encoder : IE5.5 (58 of 60 tests run so far have passed)
PASS: HTTP_Encoder : Opera identifying as IE6 (59 of 61 tests run so far have 
passed)
PASS: HTTP_Encoder : IE6 w/ "enhanced security" (60 of 62 tests run so far have 
passed)
PASS: HTTP_Encoder : deflate : uncompress possible (61 of 63 tests run so far 
have
passed)
!FAIL: HTTP_Encoder : deflate : compressed to 20.59% of original (3 of 64 tests 
run
so far have failed)
!FAIL: HTTP_Encoder : gzip : uncompress possible (4 of 65 tests run so far have 
failed)
!FAIL: HTTP_Encoder : gzip : compressed to 20.60% of original (5 of 66 tests 
run so
far have failed)
PASS: HTTP_Encoder : compress : uncompress possible (62 of 67 tests run so far 
have
passed)
!FAIL: HTTP_Encoder : compress : compressed to 20.71% of original (6 of 68 
tests run
so far have failed)
PASS: HTTP_Encoder : Vary always sent (63 of 69 tests run so far have passed)
PASS: HTTP_ConditionalGet : client has valid If-Modified-Since (64 of 70 tests 
run so
far have passed)
PASS: HTTP_ConditionalGet : client has valid If-Modified-Since with trailing
semicolon (65 of 71 tests run so far have passed)
PASS: HTTP_ConditionalGet : client has valid ETag (non-encoded version) (66 of 
72
tests run so far have passed)
PASS: HTTP_ConditionalGet : client has valid ETag (gzip version) (67 of 73 
tests run
so far have passed)
PASS: HTTP_ConditionalGet : no conditional get (68 of 74 tests run so far have 
passed)
PASS: HTTP_ConditionalGet : client has invalid ETag (69 of 75 tests run so far 
have
passed)
PASS: HTTP_ConditionalGet : client has invalid If-Modified-Since (70 of 76 
tests run
so far have passed)
PASS: JSMin : Overall (71 of 77 tests run so far have passed)
<br />
Strict Standards:  mb_strpos() [<a
href='function.mb-strpos'>function.mb-strpos</a>]: It is not safe to rely on the
system's timezone settings. Please use the date.timezone setting, the TZ 
environment
variable or the date_default_timezone_set() function. In case you used any of 
those
methods and you are still getting this warning, you most likely misspelled the
timezone identifier. We selected 'Europe/Berlin' for 'CEST/2.0/DST' instead in
/home/www/httpd/xtcReuter-Dev/htdocs/min/lib/JSMin.php on line 113<br />
<br />
Warning:  mb_strpos() [<a href='function.mb-strpos'>function.mb-strpos</a>]:
Empty delimiter in /home/www/httpd/xtcReuter-Dev/htdocs/min/lib/JSMin.php on
line 113<br />
PASS: JSMin : Quotes in RegExp literals (Issue 74) (72 of 78 tests run so far 
have
passed)
PASS: environment : DOCUMENT_ROOT should not end in trailing slash (73 of 79 
tests
run so far have passed)
PASS: environment : DOCUMENT_ROOT should pass realpath() (74 of 80 tests run so 
far
have passed)
PASS: environment : DOCUMENT_ROOT should contain this test file (75 of 81 tests 
run
so far have passed)
!FAIL: environment : PHP/server does not auto-HTTP-encode content (7 of 82 
tests run
so far have failed)

Original comment by i...@medic-world.de on 10 May 2010 at 11:02

GoogleCodeExporter commented 9 years ago 
it seems like it is a matter of gzencode

Original comment by i...@medic-world.de on 10 May 2010 at 2:06

GoogleCodeExporter commented 9 years ago 
Try placing mb_internal_encoding('8bit'); in min/config.php. In theory that 
should
make the overloaded functions behave just like the regular ones.

The test_HTTP_Encoder.php file has a PHP gzdecode function which makes *tons* of
string function calls.

Original comment by mrclay....@gmail.com on 10 May 2010 at 2:35

GoogleCodeExporter commented 9 years ago 
yes, thats what it seems like. the problems had just start with activating
mbstring.func_overload. We have had set gzip already without problems before. 
When i
disable
Minify.php
281
//                if (function_exists('gzencode')) {
//                    self::$_cache->store($cacheId . '.gz', gzencode($content,
self::$_options['encodeLevel']));
//                }

it works fine. otherwise the non zipped cache file is cutted off.

Original comment by i...@medic-world.de on 11 May 2010 at 6:52

GoogleCodeExporter commented 9 years ago 
You might try this for kicks in min/config.php:

$min_serveOptions['encodeOutput'] = false;
ob_start("ob_gzhandler");

You might have to comment out the line in Minify that sets Content-Length since 
it 
will be wrong a lot of times.

Original comment by mrclay....@gmail.com on 12 May 2010 at 12:43

GoogleCodeExporter commented 9 years ago 
comment out the content length work.

But i wont this solution, i think i did a mistake.
- we had convertet ALL files even css and js to UTF8
- we had activate function overload in the zend-server and defined UTF8
- we just had copy&paste the changes from issue 132
    this solution made a check if funtion overload is active ore not and user strlen
or mb_strlen($content, 'ASCII'). I think this not needed because the server 
have do
decide if he use multibyte funtions or not. And if i set it manualy with 
'ASCII' it
will make no sence in our case. 

as i had reported, excluding the gzencode will work, so i have to find out 
whats up
there.

i even thougt about Minify.php self::$_options['contentTypeCharset'] its 
defined in
Base.php as UTF-8. does minify convert js and css to UTF8 if userfile stored as 
ASCII?

Original comment by i...@medic-world.de on 19 May 2010 at 7:37

GoogleCodeExporter commented 9 years ago 
http://code.google.com/p/minify/source/browse/branches/2.1.4/ has a lot of work 
done
in this area. Everything is now well-tested under mbstring.func_overload, but I 
don't
know if it's gonna help w/ gzencode.

BTW, mb_internal_encoding('8bit') before any string code seems to be the key
solution. It's faster than ASCII, too.

As for contentTypeCharset, this just determines what charset is sent in the 
header.
Minify does no conversion, but, yes, ASCII *is* UTF-8, so it converts it by 
labeling
it :)

Original comment by mrclay....@gmail.com on 19 May 2010 at 1:14

GoogleCodeExporter commented 9 years ago 
i have tried 2.1.4 and it seems like i missunderstood the problem, looks like 
there
was no issue about gzencode. 2.1.4 works like a charm. i love minify, thanks a 
lot!

Original comment by i...@medic-world.de on 21 May 2010 at 7:16