search

RustPython / Parser

MIT License
72 stars 28 forks source link

Try `ThinVec` crate to reduce AST size #34

Closed MichaReiser closed 1 year ago

MichaReiser commented 1 year ago

This PR is for documentary reasons. I don't plan to land this change.

The PR removes the uses of Vec<T> with ThinVec<T> in the AST to shrink the AST node sizes. The hope was that this would improve performance, but the performance remains unchanged.

The reason why I do not recommend merging this PR is because ThinVec s cannot easily be passed to C or Python (with PyO3) which outweighs any potential memory gains from this.

Results from running ruff (https://github.com/RustPython/Parser/pull/34)

group                                      opt                                    thin
-----                                      ---                                    ----
linter/all-rules/large/dataset.py          1.01      8.5±0.10ms     4.8 MB/sec    1.00      8.4±0.09ms     4.8 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.02      2.0±0.02ms     8.2 MB/sec    1.00  1992.1±26.25µs     8.4 MB/sec
linter/all-rules/numpy/globals.py          1.01    223.4±0.81µs    13.2 MB/sec    1.00    222.2±0.40µs    13.3 MB/sec
linter/all-rules/pydantic/types.py         1.00      3.5±0.01ms     7.2 MB/sec    1.00      3.5±0.01ms     7.2 MB/sec
linter/default-rules/large/dataset.py      1.01      4.3±0.08ms     9.4 MB/sec    1.00      4.3±0.01ms     9.5 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.01    880.9±1.84µs    18.9 MB/sec    1.00    873.7±3.13µs    19.1 MB/sec
linter/default-rules/numpy/globals.py      1.03     96.3±9.46µs    30.6 MB/sec    1.00     93.5±1.29µs    31.6 MB/sec
linter/default-rules/pydantic/types.py     1.02   1886.8±7.16µs    13.5 MB/sec    1.00  1858.6±48.21µs    13.7 MB/sec
parser/large/dataset.py                    1.01      3.5±0.05ms    11.7 MB/sec    1.00      3.5±0.01ms    11.7 MB/sec
parser/numpy/ctypeslib.py                  1.00    654.0±1.27µs    25.5 MB/sec    1.00    656.3±4.92µs    25.4 MB/sec
parser/numpy/globals.py                    1.02     65.5±0.44µs    45.1 MB/sec    1.00     64.2±0.13µs    46.0 MB/sec
parser/pydantic/types.py                   1.00   1430.9±4.21µs    17.8 MB/sec    1.01  1444.3±12.34µs    17.7 MB/sec

The performance improves a bit compared to the baseline but not meaningfully.

youknowone commented 1 year ago

Related lalrpop issue https://github.com/lalrpop/lalrpop/issues/800