Feature request: Lazy loading

Floppy commented 1 week ago

Would there be (or is there already?) any way to lazy-load the matches and frequency lists, so that memory is only used when password strengths are actually checked? derailed shows the following retained memory stats:

retained memory by gem
-----------------------------------
  12628715  zxcvbn-0.1.10
    729313  activerecord-7.1.4.2
    683207  activesupport-7.1.4.2
    617131  mittsu-050afee4cbd3
    596748  opengl-bindings-1.6.14
    329826  bundled_gems
    297872  tzinfo-2.0.6
    297532  ffi-1.17.0
    288044  bindata-2.5.0
    246931  activeadmin-3.2.5
     ...

retained memory by file
-----------------------------------
   7089732  /Users/james/.rbenv/versions/3.3.5/lib/ruby/gems/3.3.0/gems/zxcvbn-0.1.10/lib/zxcvbn/matching.rb
   5498886  /Users/james/.rbenv/versions/3.3.5/lib/ruby/gems/3.3.0/gems/zxcvbn-0.1.10/lib/zxcvbn/frequency_lists.rb
    329826  /Users/james/.rbenv/versions/3.3.5/lib/ruby/3.3.0/bundled_gems.rb

As you can see, xzcvbn is the largest by far. Lazy loading would save 12MB of memory per process, which would benefit everyone! Obviously if the tables are in regular use, then keeping them loaded is the best way, but for a site with only occasional need for the strength check, there's a potential saving to be had.

formigarafa commented 1 week ago

Not yet but it a great idea. Some work have been made recently on #12 to substantially reduce these numbers. It could also help with PR #9 which is been asking to allow for configurable dictionaries.

formigarafa commented 1 week ago

This gem was aimed to be a drop-in replacement for other implementations, at least as much as possible. There is a class Zxcvbn::Tester declared at lib/zxcvbn.rb:43 which hold the data while it exists in other ports.

That is the key for implementing: lazy data loading, make it possible to free the used memory and also allow for loading alternative dictionaries without causing trouble.

formigarafa / zxcvbn-rb

Feature request: Lazy loading #15