search

ua-parser / uap-python

Python implementation of ua-parser
Apache License 2.0
561 stars 152 forks source link

Measure memory / performance if regexes are compiled in `ASCII` mode #212

Open masklinn opened 3 months ago

masklinn commented 3 months ago

Per https://docs.python.org/3/library/re.html:

  • \d Matches any Unicode decimal digit (that is, any character in Unicode character category [Nd]). This includes [0-9], and also many other digit characters.
  • \w Matches Unicode word characters; this includes all Unicode alphanumeric characters (as defined by str.isalnum()), as well as the underscore (_).

However according to https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_expressions/Character_classes (which given the reference implementation is in JS is likely the implicit reference semantics)

  • \d Digit character class escape: Matches any digit (Arabic numeral). Equivalent to [0-9]. For example, /\d/ or /[0-9]/ matches "2" in "B2 is the suite number".
  • \w Word character class escape: Matches any alphanumeric character from the basic Latin alphabet, including the underscore. Equivalent to [A-Za-z0-9_]. For example, /\w/ matches "a" in "apple", "5" in "$5.28", "3" in "3D" and "m" in "Émanuel".

This corresponds to the re.ASCII versions of the Python regexes:

  • \d Matches [0-9] if the ASCII flag is used.
  • \w Matches [a-zA-Z0-9_] if the ASCII flag is used.

This might do nothing (because regex uses library means to check those classes) or it might reduce the amount of state the regex needs to store.