ua-parser / uap-python

Python implementation of ua-parser
Apache License 2.0
561 stars 152 forks source link

Simplify codegen through the power of doing less #214

Closed masklinn closed 3 months ago

masklinn commented 3 months ago

The parameters of the matchers could be renamed to correspond to the exact attributes from regexes.yaml, allowing directly splatting dict literals in the generated code instead of generating sequences of positional attributes. e.g. instead of generating

UserAgentMatcher("foo", "bar", None, None, None, None)

generate

UserAgentMatcher(**{
    "regex": "foo",
    "family_replacement": "bar",
    "v1_replacement ": None,
    "v2_replacement ": None,
    "v3_replacement ": None,
    "v4_replacement ": None,
)

this is a much easier serialisation. Although it does take more space in the generated file.

masklinn commented 3 months ago

Closing, after investigation this generates a lot more bytecode:

>>> dis.dis("foo(bar)")
  0           0 RESUME                   0

  1           2 PUSH_NULL
              4 LOAD_NAME                0 (foo)
              6 LOAD_NAME                1 (bar)
              8 CALL                     1
             16 RETURN_VALUE
>>> dis.dis("foo(**{'foo': 'bar'})")
  0           0 RESUME                   0

  1           2 PUSH_NULL
              4 LOAD_NAME                0 (foo)
              6 LOAD_CONST               2 (())
              8 BUILD_MAP                0
             10 LOAD_CONST               0 ('foo')
             12 LOAD_CONST               1 ('bar')
             14 BUILD_MAP                1
             16 DICT_MERGE               1
             18 CALL_FUNCTION_EX         1
             20 RETURN_VALUE

and the performance hit is genuinely staggering from all the additional intermediate stuff:

> python -mtimeit -s 'foo = lambda foo: None' 'foo("bar")'
10000000 loops, best of 5: 21.6 nsec per loop
> python -mtimeit -s 'foo = lambda foo: None' 'foo(**{"foo":"bar"})'
2000000 loops, best of 5: 136 nsec per loop

this is using Python 3.12.4 on an M1P.