p5pRT
commented
10 years ago From @hvds
Created by @hvds
I've been experimenting with an attempt to take a SQL grammar expressed in BNF and convert it (programmatically) into something that can parse SQL with it as a Regexp::Grammars (v1.035) grammar.
The code below is (60%) cut down from an interim stage in that process; this reaches about 10MB process size under perl-5.16.3; under perl-5.20.0 it grows to over 1GB. Cutting down the grammar rule by rule does gradually reduce the memory use\, but it remains a high multiple of the memory use under perl-5.16.3\, and I've not yet found any smoking gun; I've included the full 200-odd lines here rather than risk eliding something important.
Damain and I are looking into it\, but he suggested I perlbug it as a heads-up of a possible problem in 5.20\, likely of interest to davem as potentially relating to regexp engine changes.
zen% ulimit -v # I've set a 1GB process-size limit 1000000 zen% /usr/bin/time /opt/perl-5.16.3/bin/perl ./t0 # top(1) shows peak 10MB VIRT ok 8.52user 0.01system 0:08.54elapsed 99%CPU (0avgtext+0avgdata 34816maxresident)k 0inputs+0outputs (0major+2331minor)pagefaults 0swaps zen% /usr/bin/time /opt/perl-5.20.0/bin/perl ./t0 Out of memory! Command exited with non-zero status 1 41.59user 2.10system 0:43.83elapsed 99%CPU (0avgtext+0avgdata 3641344maxresident)k 0inputs+0outputs (0major+228082minor)pagefaults 0swaps zen% cat t0 #!/opt/perl-5.20.0/bin/perl use strict; use warnings; use Regexp::Grammars;
my $g = qr{ ^ \<query_specification> $
\<rule: simple_Latin_letter> \<simple_Latin_upper_case_letter> | \<simple_Latin_lower_case_letter>
\<token: simple_Latin_upper_case_letter> A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z
\<token: simple_Latin_lower_case_letter> a | b | c | d | e | f | g | h | i | j | k | l | m | n | o | p | q | r | s | t | u | v | w | x | y | z
\<token: digit> 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
\<token: double_quote> \"
\<token: left_paren> \(
\<token: right_paren> \)
\<token: asterisk> \*
\<token: plus_sign> \+
\<token: comma> \\,
\<token: minus_sign> \-
\<token: period> \.
\<token: solidus> \/
\<token: less_than_operator> \\<
\<token: equals_operator> \=
\<token: greater_than_operator> \>
\<token: question_mark> \?
\<token: underscore> _
\<rule: regular_identifier> \<identifier_body>
\<rule: identifier_body> \<identifier_start> (?:\
}x; print "ok\n"; zen%
Migrated from rt.perl.org#122283 (status was 'resolved')
Searchable as RT122283$