Utilize PhpToken::tokenize() when available

nette / utils

🛠 Lightweight utilities for string & array manipulation, image handling, safe JSON encoding/decoding, validation, slug or strong password generating etc.

https://doc.nette.org/utils

Other

1.98k stars 147 forks source link

Utilize PhpToken::tokenize() when available #296

Closed staabm closed 1 year ago

staabm commented 1 year ago

bug fix / new feature?
BC break? no
doc PR: nette/docs#???

I can see Rector to bottleneck on token_get_all.

do you think we could backport this change to PhpToken::tokenize to 3.x so libs which need php7 compat can still benefit from the memory/perf improvements of PhpToken::tokenize when run on php8?

php7 compatible backport of https://github.com/nette/utils/commit/8e2e71cd702a59ba4e786bb031135583e2f1ef23

staabm commented 1 year ago

did some measures locally and it seems the differences are negligible