Add a regex benchmark suite

[GoogleCodeExporter]

Python currently doesn't have a good-quality regex benchmark suite that can
be run automatically, have statistics drawn from it, etc. We need such a
thing before starting work on regex performance.

Possible resources:
- Fredrik Lundh's original benchmarks for SRE:
http://mail.python.org/pipermail/python-dev/2000-August/007797.html
- V8's JS regex benchmarks:
http://v8.googlecode.com/svn/data/benchmarks/v3/regexp.js

Ideally we would do a search of the Python regexes in Google Code Search or
similar corpus and distill some representative set from them. V8's may be
good enough, though.

Original issue reported on code.google.com by collinw on 14 Apr 2009 at 11:37

[GoogleCodeExporter]

Original comment by collinw on 27 May 2009 at 11:07

• Added labels: Type-Enhancement

[GoogleCodeExporter]

Original comment by collinw on 29 May 2009 at 12:09

• Added labels: StarterProject

[GoogleCodeExporter]

Original comment by collinw on 29 May 2009 at 4:13

• Added labels: Priority-Medium, Release-2009Q3

[GoogleCodeExporter]

r615 adds V8's regex benchmarks to perf.py. I still want to include the 
original SRE
benchmarks mentioned in
http://mail.python.org/pipermail/python-dev/2000-August/007797.html.

Original comment by collinw on 8 Jun 2009 at 8:48

[GoogleCodeExporter]

As of r623, "perf.py -b regex_effbot" runs the regex benchmarks listed in the 
link 
above. Thanks to David Laing for the patch!

There's also now a regex benchmark group that runs both regex_v8 and 
regex_effbot. "perf.py -b regex" will do the trick.

The mailing list thread cited above lists some macrobenchmarks that I'd still 
like to 
include; regex_v8 is based on popularity of JavaScript regexes (which makes me 
a 
little nervous), and regex_effbot is pretty micro. Some real-app 
macrobenchmarks 
would make me more confident.

Original comment by collinw on 21 Jun 2009 at 2:56

[GoogleCodeExporter]

I'd also like a regex_compile benchmark that stresses regex compilation time. 
If we start 
compiling regexes to machine code, we need to know how much that will hurt us.

This could be as simple as importing regex_v8 and regex_effbot, collecting the 
regex 
strings from those modules, then compiling them all several times.

Original comment by collinw on 15 Jul 2009 at 8:41

[GoogleCodeExporter]

Added regex_compile in r782, which captures the regexes used by regex_v8 and 
regex_effbot.

Between regex_compile, regex_effbot and regex_v8, that should be good enough to 
get 
started optimizing the regex engine. I'd still like some macro-level regex 
benchmarks, 
but that's low-priority given what we already have in place.

Original comment by collinw on 28 Jul 2009 at 9:07

• Added labels: Priority-Low
• Removed labels: Priority-Medium, Release-2009Q3