FGRibreau / mailchecker

:mailbox: Cross-language temporary (disposable/throwaway) email detection library. Covers 55 734+ fake email providers.
https://twitter.com/FGRibreau
MIT License
1.64k stars 258 forks source link

Add Ruby frozen_string_literal pragma to avoid ~75MB of String allocations #445

Closed owst closed 8 months ago

owst commented 10 months ago

I noticed that this gem makes a large number of allocations - if we add the frozen_string_literal pragma the allocations are vastly reduced. (See this for a good explanation why)

In summary, for me using Ruby 3.2.3, before this change I see ~85MB of allocations and afterwards I see ~10MB of allocations. The following script demonstrates the improvement:

#!/usr/bin/env bash
set -euv

gem unpack ruby-mailchecker -v 6.0.1
gem install heap-profiler -v 0.7.0

cat << EOF > profile.rb
require 'heap-profiler'

HeapProfiler.report("profile_#{ARGV.first}") do
  require_relative "ruby-mailchecker-6.0.1/platform/ruby/mail_checker"

  p MailChecker::BLACKLIST.size
end
EOF

ruby profile.rb before

file=ruby-mailchecker-6.0.1/platform/ruby/mail_checker.rb

echo '# frozen_string_literal: true' | cat - "$file" > temp && mv temp "$file"

ruby profile.rb after

heap-profiler profile_before | head
heap-profiler profile_after | head

For my run just now the following was printed:

heap-profiler profile_before | head
Total allocated: 89.90 MB (225505 objects)
Total retained: 5.19 MB (56057 objects)

allocated memory by gem
-----------------------------------
  87.59 MB  other
   2.31 MB  3.2.3/lib
   8.17 kB  rubygems
  320.00 B  heap-profiler-0.7.0

heap-profiler profile_after | head
Total allocated: 11.98 MB (169331 objects)
Total retained: 5.19 MB (56057 objects)

allocated memory by gem
-----------------------------------
   9.67 MB  other
   2.31 MB  3.2.3/lib
   8.13 kB  rubygems
  320.00 B  heap-profiler-0.7.0