mozilla / fx-private-relay

Keep your email safe from hackers and trackers. Make an email alias with 1 click, and keep your address to yourself.
https://relay.firefox.com
Other
1.44k stars 168 forks source link

MPP-3827: Optimize badwords and blocklist checks #4822

Closed jwhitlock closed 3 days ago

jwhitlock commented 4 days ago

There are a few ways to reject a term provided by the user:

  1. A short term, 4 characters or less, is in the bad words list.
  2. A long term, 5 characters or more, contains a bad word that is 5 characters or more.
  3. A word is in the blocklist

This PR uses a Python set() for the first and third checks, which should be faster than iterating over the whole list. It makes other changes to word list loading an related tests.

I used this file to time the differences:

https://gist.github.com/jwhitlock/b470221e18b61a6647dd5684f26a427d

I placed this at emails/timeit.py, and ran ./manage.py shell:

>>> from emails.timeit import timer_test
>>> timer_test()
Checking that tests pass...
Old code took: 0.013196
New code took: 0.009787
Speedup is 1.3

This is faster, but the old version runs 1,000,000 times in 13ms, versus 10ms for the new version. This should have no noticable impact in production, so a valid review can reject it because it makes the code harder to understand.