Closed MichaReiser closed 1 year ago
This PR tries to reduce the number of elements that we copy by
vec.extend
vec.insert(0, element)
map|x| x.1)
I hoped that this would improve performance more but it remains mostly unchanged
group base opt ----- ---- --- parser/large/dataset.py 1.00 3.5±0.01ms 11.7 MB/sec 1.00 3.5±0.05ms 11.7 MB/sec parser/numpy/ctypeslib.py 1.02 664.1±1.66µs 25.1 MB/sec 1.00 654.0±1.27µs 25.5 MB/sec parser/numpy/globals.py 1.02 67.1±0.56µs 44.0 MB/sec 1.00 65.5±0.44µs 45.1 MB/sec parser/pydantic/types.py 1.00 1416.3±19.83µs 18.0 MB/sec 1.01 1430.9±4.21µs 17.8 MB/sec
This PR tries to reduce the number of elements that we copy by
vec.extend
vec.insert(0, element)
map|x| x.1)
to ignore some tokensPerformance
I hoped that this would improve performance more but it remains mostly unchanged