Performance degradation when using big tt-muncher `macro_rules`

[Robbepop]

I recently noticed a big performance degradation regarding rust-analyzer in one of my projects: https://github.com/paritytech/wasmi

The problem is that rust-analyzer consumes much more memory than it has used to and also very often get stuck and shows incorrect warnings and won't finish analysis of the crate and therefore is pretty much unusable for the project. I noticed a very similar behavior of rust-analyzer in the https://github.com/bytecodealliance/wasm-tools repository.

After some backtracking I found that the performance regressions started after merging these PRs:

• https://github.com/paritytech/wasmi/pull/644
• https://github.com/paritytech/wasmi/pull/645

What do the PRs do? They refactor use of wasmparser's for_each_operator macro which is supposed to be used as tt-muncher macro in order to perform operations on the over 500 different Wasm operators.

My best guess at what portion of wasmi causes the choking for rust-analyzer is probably these code segments:

• https://github.com/paritytech/wasmi/blob/3e6d0d63846ff7c3000c0934e00ad0d34ea6a10f/crates/wasmi/src/engine/func_builder/mod.rs#L177
• https://github.com/paritytech/wasmi/blob/3e6d0d63846ff7c3000c0934e00ad0d34ea6a10f/crates/wasmi/src/engine/func_builder/translator.rs#L642

Note that in particular in https://github.com/paritytech/wasmi/pull/645 we had to increase macro recursion limit in lib.rs:

#![recursion_limit = "750"]

Sometimes I succeed in getting rust-analyzer unstuck by performing a cargo clean on my workspace. However, this is notably not a very neat solution for obvious reasons.

[lnicola]

There are a couple of open issues about MBE performance. I suspect one of the problems is that we convert too often from token trees to a linear form.