search

FuelLabs / sway

🌴 Empowering everyone to build reliable and efficient smart contracts.
https://docs.fuel.network/docs/sway/
Apache License 2.0
62.58k stars 5.37k forks source link

forc-lsp consuming a ton of memory #5645

Open diyahir opened 9 months ago

diyahir commented 9 months ago
Screenshot 2024-02-22 at 15 31 33
diyahir commented 9 months ago

For context, I have a project with 20 contracts 😭

diyahir commented 9 months ago

Watch at 2x speed, after a forc-lsp stalls which happens every time I change some code in a big codebase I have to close vscode and reopen it to start a new forc-lsp

The stalled processes never die and I need to do this, roughly every time I change code, but upon reopening it works fine, but it cannot handle any code changes

https://github.com/FuelLabs/sway/assets/32445955/d158c8bd-301d-46b3-8868-2864747b53b7

JoshuaBatty commented 8 months ago

Hi @diyahir can you please provide information about the version of forc-lsp you are using?

diyahir commented 8 months ago

forc : 0.49.2

JoshuaBatty commented 8 months ago

Thanks @diyahir. Really appreciate the video as well. It seems like there are a few things going on here.

When you save, the formatter is taking a very long time to complete, this then triggers an LSP event to re-parse the entire workspace. The project is so big that the server is left in a hanging state if the IDE is restarted before it can finish.

We recently enabled module level caching but it seems like this isn't having much of an affect on your 20+ contract workspace.

It seems like we should look at 2 things:

  1. trying to reduce the overall memory foot print
  2. enable more granular caching, probably at the file level.

I'm not sure I can provide any good news for you in the short term, without these 2 things implemented the server will be slow when dealing with large workspaces. I can promise we will be actively looking at addressing these issues asap.

Please feel free to continue reporting any other issues, they are really helpful. Thanks!

diyahir commented 8 months ago

The curious thing is that upon restarting the fresh forc-lsp state is fully able to go through the entire state and provide the relevant errors, so it's really from the live transition that messes everything up, a good interm solution for me at least would be a manual restart command for the forc-lsp, just to avoid having to close vscode and re-open, this is of course a bandaid but if it's a small lift would be appreciated!

JoshuaBatty commented 8 months ago

Ok we can look at adding a command for restarting the server. Just curious, does turning off format on save feature help the responsiveness?

diyahir commented 8 months ago

Ok we can look at adding a command for restarting the server. Just curious, does turning off format on save feature help the responsiveness?

No this did not fix the issue, same behavior

sdankel commented 8 months ago

The "Reload Window" command restarts LSP, if that's helpful.

diyahir commented 8 months ago

Any updates on this?

I suspect auditors will run into the same issue and have less patience when working with large repos

JoshuaBatty commented 8 months ago

This work will most likely be ongoing for the next few months. If you want to add me to your repo then I can take a look at what seem to be the easy low hanging fruits we can fix to get it working for you. Just an idea, all good if you would prefer to keep the repo closed. Otherwise, i'll continue to work on points 1 & 2 mentioned above. Each one is not trivial and will likely take some time to get implemented.

diyahir commented 7 months ago

Just an update, I got a new computer unrelated to this it's an M3 MB, 16 gbs of ram, and i'm working in repo with 3 contracts, and there are still massive LSP slowness issues

It's really unusable, I tried the suggestions, disabling github copilot, and format on save, and still it's painfully slow

Maybe @JoshuaBatty we can hop on a quick code session to see if see some code pattern i did that is causing this?

JoshuaBatty commented 7 months ago

Hi @diyahir, sure i'm happy to jump on a call. There are also some discussion going on in this thread that might be worth looking at. https://github.com/FuelLabs/sway-vscode-plugin/issues/172

The temp solution at the bottom might help with the speed but it will make the RAM usage worse. It's next on my todo list after fixing this garbage collection compiler bug.

Nazeeh21 commented 5 months ago

Is there any updates on this?

JoshuaBatty commented 5 months ago

The memory issue is not going to be trivial to fix. This is going to take substantial effort with some compiler redesign to support correctly. The main problem here is that for large workspaces like the FUSD repo (19+) projects. We need to store a unique engines for each project, even though each Engines contains duplicates of std, core and any dependancies. There is no way to share these types across projects within the same workspace with the way the compiler is currently. I can't see this being solved in the near term. A realistic expectation would be Q4 this year.

JoshuaBatty commented 4 months ago

We have a temp solution for this coming https://github.com/FuelLabs/sway/pull/6239

Basically we only keep the last 4 opened projects in memory so we can cap the memory usage growing indefinitely. We have some other fixes that will help with memory usage on the way also.