Rust Analyzer doesn't work with large minimized files

[TCROC]

rust-analyzer version:

rust-analyzer version: 0.3.1665-standalone

rustc version:

rustc 1.72.1 (d5c2e9c34 2023-09-13)

Example Project: https://github.com/TCROC/rust-analyzer-gdnative-issue-example

The Godot-Rust gdnative extension here: https://github.com/godot-rust/gdnative

Gdnative generates bindings for the Godot engine. It does this in 1 of 3 ways:

1. Generating all of the bindings in a single minimized file. This results in faster compile times when building against the bindings. Rust analyzer does not work with this file.

2. Generating all of the bindings per class / struct in separate minimized files. This seems to have similar build times. Rust analyzer does work with this file.

3. Generating all of the bindings per class / struct in separate files and formatting those files. Rust analyzer works on these files, but the compile times are very slow.

Fortunately I can use options 2 and 3 as workarounds for now. But I feel that rust analyzer should work on a single minified file as well. The linked repo contains examples for 1 (the issue) and 2 (the best workaround for now).

I created an example project here with a README explaining how to reproduce:

https://github.com/TCROC/rust-analyzer-gdnative-issue-example

Here is the README for convenient reading without clicking the above link:

broken-example

This folder illustrates the issue. If you open it in vs code: https://github.com/microsoft/vscode , you will see that the auto completions and comment hovers are not working. Here is a screenshot to demonstrate:

Cargo.toml

[package]
name = "broken-example"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
gdnative = "0.11.3"
gdnative-bindings = "0.11.3"

working-example

This folder illustrates a workaround that can be applied with the gdnative-bindings library. If you enable the "one-class-one-file" feature:

gdnative-bindings = { version = "0.11.3", features = [ "one-class-one-file" ] }

Then the generated rust code will be broken up into multiple files instead of compressed in one file. Now the rust-analyzer works properly:

Cargo.toml

[package]
name = "working-example"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
gdnative = "0.11.3"
gdnative-bindings = { version = "0.11.3", features = [ "one-class-one-file" ] }

Now I do think rust-analyzer should be able to handle this. I do think it is likely a bug on the rust-analyzer side. Fortunately, there is a workaround for the gdnative crate. Other crates may have issues though if they generate code and do not break the code apart into different files.

I hope you adventuring lads find this project useful in your debugging journey! :)

END OF README ========================

Thanks again for the help and let me know if you need anything from me!

[TCROC]

And I put in an issue with gdnative to default to the option 2 workaround for now while the real bug here is being looked into:

https://github.com/godot-rust/gdnative/issues/1053

[Veykril]

Looks like the bindings are include!d, so I assume that the mega file blows the expansion token limit which we then reject. cc https://github.com/rust-lang/rust-analyzer/issues/10855

[lnicola]

I hope you adventuring lads find this project useful in your debugging journey! :)

It's dangerous to go alone! Take this.

On a serious note, that's right, increasing TOKEN_LIMIT makes it work. But didn't I bump one of these limits recently?

[TCROC]

Looks like the bindings are include!d, so I assume that the mega file blows the expansion token limit which we then reject. cc #10855

Awesome! So is this as simple as doing 1 of 2 things:

1. Adding a setting to rust-analyzer for token limits that can be configured in VS Code settings

2. Just remove the token limit? Or at least remove it by default and put it there for users to add if they run into a situation where a project is too big and balloons their memory up too much?

Now that I think about it, I actually like the idea of defaulting to no token limit with a config option to enable it. It would make sense as we are all swimming in memory today and the rare edge cases of people that run out of memory can error out with a helpful message like:

"Try setting a token limit as it appears you ran out of memory."

Then point to rust-analyzer docs on how to do it or link to the vs code setting.

[TCROC]

It's dangerous to go alone! Take this.

I understood that reference 😎

[lnicola]

Fixed in #15819.