Ruby bindings to RE2, a "fast, safe, thread-friendly alternative to backtracking regular expression engines like those used in PCRE, Perl, and Python".
Current version: 2.14.0
Bundled RE2 version: libre2.11 (2024-07-02)
RE2('h.*o').full_match?("hello") #=> true
RE2('e').full_match?("hello") #=> false
RE2('h.*o').partial_match?("hello") #=> true
RE2('e').partial_match?("hello") #=> true
RE2('(\w+):(\d+)').full_match("ruby:1234")
#=> #<RE2::MatchData "ruby:1234" 1:"ruby" 2:"1234">
While recent versions of Ruby have improved defences against regular expression denial of service (ReDoS) attacks, it is still possible for users to craft malicious patterns that take a long time to process by using syntactic features such as back-references, lookaheads and possessive quantifiers. RE2 aims to eliminate ReDoS by design:
Safety is RE2's raison d'être.
RE2 was designed and implemented with an explicit goal of being able to handle regular expressions from untrusted users without risk. One of its primary guarantees is that the match time is linear in the length of the input string. It was also written with production concerns in mind: the parser, the compiler and the execution engines limit their memory usage by working within a configurable budget – failing gracefully when exhausted – and they avoid stack overflow by eschewing recursion.
— Why RE2?
Install re2 as a dependency:
# In your Gemfile
gem "re2"
# Or without Bundler
gem install re2
Include in your code:
require "re2"
Full API documentation automatically generated from the latest version is available at https://mudge.name/re2/.
While re2 uses the same naming scheme as Ruby's built-in regular expression
library (with Regexp
and
MatchData
), its API is slightly
different:
[!WARNING] RE2's regular expression syntax differs from PCRE and Ruby's built-in
Regexp
library, see the official syntax page for more details.
The core class is RE2::Regexp
which
takes a regular expression as a string and compiles it internally into an RE2
object. A global function RE2
is available to concisely compile a new
RE2::Regexp
:
re = RE2('(\w+):(\d+)')
#=> #<RE2::Regexp /(\w+):(\d+)/>
re.ok? #=> true
re = RE2('abc)def')
re.ok? #=> false
re.error #=> "missing ): abc(def"
[!TIP] Note the use of single quotes when passing the regular expression as a string to
RE2
so that the backslashes aren't interpreted as escapes.
When compiling a regular expression, an optional second argument can be used to change RE2's default options, e.g. stop logging syntax and execution errors to stderr with log_errors
:
RE2('abc)def', log_errors: false)
See the API documentation for RE2::Regexp#initialize
for all the available options.
There are two main methods for matching: RE2::Regexp#full_match?
requires the regular expression to match the entire input text, and RE2::Regexp#partial_match?
looks for a match for a substring of the input text, returning a boolean to indicate whether a match was successful or not.
RE2('h.*o').full_match?("hello") #=> true
RE2('e').full_match?("hello") #=> false
RE2('h.*o').partial_match?("hello") #=> true
RE2('e').partial_match?("hello") #=> true
[!TIP] Only extract the number of submatches you need as performance is improved with fewer submatches (with the best performance when avoiding submatch extraction altogether).
Both matching methods have a second form that can extract submatches as RE2::MatchData
objects: RE2::Regexp#full_match
and RE2::Regexp#partial_match
.
m = RE2('(\w+):(\d+)').full_match("ruby:1234")
#=> #<RE2::MatchData "ruby:1234" 1:"ruby" 2:"1234">
m[0] #=> "ruby:1234"
m[1] #=> "ruby"
m[2] #=> "1234"
m = RE2('(\w+):(\d+)').full_match("r")
#=> nil
RE2::MatchData
supports retrieving submatches by numeric index or by name if present in the regular expression:
m = RE2('(?P<word>\w+):(?P<number>\d+)').full_match("ruby:1234")
#=> #<RE2::MatchData "ruby:1234" 1:"ruby" 2:"1234">
m["word"] #=> "ruby"
m["number"] #=> "1234"
They can also be used with Ruby's pattern matching:
case RE2('(\w+):(\d+)').full_match("ruby:1234")
in [word, number]
puts "Word: #{word}, Number: #{number}"
else
puts "No match"
end
# Word: ruby, Number: 1234
case RE2('(?P<word>\w+):(?P<number>\d+)').full_match("ruby:1234")
in word:, number:
puts "Word: #{word}, Number: #{number}"
else
puts "No match"
end
# Word: ruby, Number: 1234
By default, both full_match
and partial_match
will extract all submatches into the RE2::MatchData
based on the number of capturing groups in the regular expression. This can be changed by passing an optional second argument when matching:
m = RE2('(\w+):(\d+)').full_match("ruby:1234", submatches: 1)
=> #<RE2::MatchData "ruby:1234" 1:"ruby">
[!WARNING] If the regular expression has no capturing groups or you pass
submatches: 0
, the matching method will behave like itsfull_match?
orpartial_match?
form and only returntrue
orfalse
rather thanRE2::MatchData
.
If you want to repeatedly match regular expressions from the start of some input text, you can use RE2::Regexp#scan
to return an Enumerable
RE2::Scanner
object which will lazily consume matches as you iterate over it:
scanner = RE2('(\w+)').scan(" one two three 4")
scanner.each do |match|
puts match.inspect
end
# ["one"]
# ["two"]
# ["three"]
# ["4"]
RE2::Set
represents a collection of
regular expressions that can be searched for simultaneously. Calling
RE2::Set#add
with
a regular expression will return the integer index at which it is stored within
the set. After all patterns have been added, the set can be compiled using
RE2::Set#compile
,
and then
RE2::Set#match
will return an array containing the indices of all the patterns that matched.
set = RE2::Set.new
set.add("abc") #=> 0
set.add("def") #=> 1
set.add("ghi") #=> 2
set.compile #=> true
set.match("abcdefghi") #=> [0, 1, 2]
set.match("ghidefabc") #=> [2, 1, 0]
[!WARNING] Note RE2 only supports UTF-8 and ISO-8859-1 encoding so strings will be returned in UTF-8 by default or ISO-8859-1 if the
:utf8
option for theRE2::Regexp
is set tofalse
(any other encoding's behaviour is undefined).
For backward compatibility: re2 won't automatically convert string inputs to the right encoding so this is the responsibility of the caller, e.g.
# By default, RE2 will process patterns and text as UTF-8
RE2(non_utf8_pattern.encode("UTF-8")).match(non_utf8_text.encode("UTF-8"))
# If the :utf8 option is false, RE2 will process patterns and text as ISO-8859-1
RE2(non_latin1_pattern.encode("ISO-8859-1"), utf8: false).match(non_latin1_text.encode("ISO-8859-1"))
This gem requires the following to run:
It supports the following RE2 ABI versions:
Where possible, a pre-compiled native gem will be provided for the following platforms:
aarch64-linux
and arm-linux
(requires glibc 2.29+)x86-linux
and x86_64-linux
(requires glibc 2.17+)x86_64-darwin
and arm64-darwin
x64-mingw32
and x64-mingw-ucrt
SHA256 checksums are included in the release notes for each version and can be checked with sha256sum
, e.g.
$ gem fetch re2 -v 2.14.0
Fetching re2-2.14.0-arm64-darwin.gem
Downloaded re2-2.14.0-arm64-darwin
$ sha256sum re2-2.14.0-arm64-darwin.gem
3c922d54a44ac88499f6391bc2f9740559381deaf7f4e49eef5634cf32efc2ce re2-2.14.0-arm64-darwin.gem
GPG signatures are attached to each release (the assets ending in .sig
) and can be verified if you import our signing key 0x39AC3530070E0F75
(or fetch it from a public keyserver, e.g. gpg --keyserver keyserver.ubuntu.com --recv-key 0x39AC3530070E0F75
):
$ gpg --verify re2-2.14.0-arm64-darwin.gem.sig re2-2.14.0-arm64-darwin.gem
gpg: Signature made Fri 2 Aug 12:39:12 2024 BST
gpg: using RSA key 702609D9C790F45B577D7BEC39AC3530070E0F75
gpg: Good signature from "Paul Mucur <mudge@mudge.name>" [unknown]
gpg: aka "Paul Mucur <paul@ghostcassette.com>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 7026 09D9 C790 F45B 577D 7BEC 39AC 3530 070E 0F75
The fingerprint should be as shown above or you can independently verify it with the ones shown in the footer of https://mudge.name.
ruby
platform gem[!WARNING] We strongly recommend using the native gems where possible to avoid the need for compiling the C++ extension and its dependencies which will take longer and be less reliable.
If you wish to compile the gem, you will need to explicitly install the ruby
platform gem:
# In your Gemfile with Bundler 2.3.18+
gem "re2", force_ruby_platform: true
# With Bundler 2.1+
bundle config set force_ruby_platform true
# With older versions of Bundler
bundle config force_ruby_platform true
# Without Bundler
gem install re2 --platform=ruby
You will need a full compiler toolchain for compiling Ruby C extensions (see
Nokogiri's "The Compiler
Toolchain")
plus the toolchain required for compiling the vendored version of RE2 and its
dependency Abseil which includes CMake, a compiler
with C++14 support such as clang 3.4 or
gcc 5 and a recent version of
pkg-config. On
Windows, you'll also need pkgconf 2.1.0+ to avoid undefined reference
errors when attempting to
compile Abseil.
If you already have RE2 installed, you can instruct the gem not to use its own vendored version:
gem install re2 --platform=ruby -- --enable-system-libraries
# If RE2 is not installed in /usr/local, /usr, or /opt/homebrew:
gem install re2 --platform=ruby -- --enable-system-libraries --with-re2-dir=/path/to/re2/prefix
Alternatively, you can set the RE2_USE_SYSTEM_LIBRARIES
environment variable instead of passing --enable-system-libraries
to the gem
command.
RE2::MatchData#begin
and RE2::MatchData#end
.RE2::Regexp#scan
, contributing support for libre2.11
(2023-07-01) and for vendoring RE2 and abseil and compiling native gems in
2.0.utf8
encoding option in RE2.RE2::Scanner#scan
with an invalid regular expression.RE2::Set
.TypedData
API and the resulting garbage collection
improvements in 2.4.0.All issues and suggestions should go to GitHub Issues.
This library is licensed under the BSD 3-Clause License, see LICENSE.txt
.
Copyright © 2010, Paul Mucur.
The source code of RE2 is distributed in the ruby
platform gem. This code is licensed under the BSD 3-Clause License, see LICENSE-DEPENDENCIES.txt
.
The source code of Abseil is distributed in the ruby
platform gem. This code is licensed under the Apache License 2.0, see LICENSE-DEPENDENCIES.txt
.