search

ballerina-platform / ballerina-lang

The Ballerina Programming Language
https://ballerina.io/
Apache License 2.0
3.68k stars 751 forks source link

[Bug]: When opening Ballerina projects with large ballerina files the VS Code plugin crashes #43104

Open ayeshLK opened 3 months ago

ayeshLK commented 3 months ago

Description

$subject

Following error log was found in the VS Code logs:

[Error - 5:44:40 PM] Stopping server failed
Error: Stopping the server timed out
    at /home/ayesh/.vscode/extensions/wso2.ballerina-4.7.2/dist/extension.js:2:1065110
[Error - 5:44:40 PM] Stopping server failed
Error: Stopping the server timed out
    at /home/ayesh/.vscode/extensions/wso2.ballerina-4.7.2/dist/extension.js:2:1065110
[Error - 6:03:11 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:03:11 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:13:32 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:13:32 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:13:32 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:13:32 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:13:32 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:13:51 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:13:51 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:13:51 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:13:51 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:13:51 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:13:51 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
ballerina: Oh no, something really went wrong. Bad. Sad.

We appreciate it if you can report the code that broke Ballerina in
https://github.com/ballerina-platform/ballerina-lang/issues with the
log you get below and your sample code.

We thank you for helping make us better.

[Error - 6:14:11 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[2024-07-12 18:14:11,133] SEVERE {b7a.log.crash} - Java heap space 
java.lang.OutOfMemoryError: Java heap space

[Error - 6:14:11 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:14:11 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:14:11 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:14:11 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:14:11 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:14:11 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:14:11 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:14:11 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:14:11 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:14:11 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:14:11 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:14:11 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.
[Error - 6:14:51 PM] Client Ballerina LS Client: connection to server is erroring. Shutting down server.

Following heapdump was found in the project [1]

[1] - https://drive.google.com/file/d/18dKWEe9Yzu63VqpplLQK-aH1lf102Lcp/view?usp=sharing

Steps to Reproduce

Try opening the client.bal in the following project using VS Code.

stripe_new.zip

Affected Version(s)

No response

OS, DB, other environment details and versions

No response

Related area

-> Compilation

Related issue(s) (optional)

No response

Suggested label(s) (optional)

No response

Suggested assignee(s) (optional)

No response

nipunayf commented 3 months ago

Related to https://github.com/ballerina-platform/ballerina-lang/issues/42612

ayeshLK commented 3 months ago

In addition to the above when I import the stripe package into another ballerina project (which only has few lines of code), and try to open it via VS Code the Ballerina VS Code plugin crashes with a similar behaviour. IMO this is a critical issue which we should fix as soon as possible.

ayeshLK commented 3 months ago

When divide the types.bal into multiple files [1] the LS OOM was fixed. But, when I try to locally publish this connector and use it in a Ballerina project the LS OOM still happens. The heap-dump can be found here [2]

I can see the following error notification in VS Code. vs-code-crash

[1] - https://github.com/ballerina-platform/module-ballerinax-stripe/pull/4 [2] - https://drive.google.com/file/d/1Fj5KrrA_u9w-AHygHCZDGoANlImzjNtx/view?usp=sharing

ayeshLK commented 3 months ago

@KavinduZoysa currently I am blocked in this issue and unable to resolve this [1] internal support ticket. Do we have any progress on this or do you guys have a workaround ?

[1] - https://github.com/wso2-enterprise/internal-support-ballerina/issues/719

KavinduZoysa commented 3 months ago

@ayeshLK, still we are trying to find the root cause for this issue. Sometimes we see a GSON array with the size of 300MB-500MB and sometimes, we observe the heap is mostly allocated for the project context. Currently, we are investigating both aspects to find the exact root cause. We will update on the progress.

As a workaround, we can divide the types.bal file into multiple files launch vs code. But if we import the same library in a different project, we are getting OOM.

mindula commented 3 months ago

In addition to what @KavinduZoysa has mentioned, we increased the memory to 2GB, and we didn't encounter the crash anymore.

mindula commented 3 months ago

I tried importing the Stripe module after building it locally and did not encounter any OOM issues. We are still investigating the OOM issue with large files and will provide updates on our progress.

mindula commented 2 months ago

When we analysed the heap dump, we identified an ArrayList with 2,927 elements (matching the number of types in the types.bal file) as the main suspect.

Screenshot 2024-08-12 at 17 00 18

After examining the JsonObjects, we observed that they contained key value pairs representing the syntax tree, as shown in the below image.

Screenshot 2024-08-12 at 16 42 58

After debugging the code, we discovered that these JsonObjects were created in the Diagram.util class. After commenting out the relevant lines and returning an empty json object instead, we no longer encountered the OOM issue, and all Language Server functionalities worked as expected.

It appears that the Diagram.util class represents every token with a JsonObject. @kanushka @hevayo

kanushka commented 2 months ago

After debugging the code, we discovered that these JsonObjects were created in the Diagram.util class. After commenting out the relevant lines and returning an empty json object instead, we no longer encountered the OOM issue, and all Language Server functionalities worked as expected.

It appears that the Diagram.util class represents every token with a JsonObject. @kanushka @hevayo

This JSON syntax tree is used to generate the Ballerina low-code diagram. I think we can deprecate this generation since the new Ballerina extension will use another LS API to generate the low-code diagram.

kanushka commented 2 months ago

This JSON syntax tree is used to generate the Ballerina low-code diagram. I think we can deprecate this generation since the new Ballerina extension will use another LS API to generate the low-code diagram.

I'm sorry, we cannot deprecate this generation since we are using this functionality in a few more places.

The initial language service shutdown issue will occur due to code lens generation logic. Ballerina extension uses the getSyntaxTree API and traverses visitors to find possible code lenses. Since the file is too bulky, getSyntaxTree will result in an out-of-memory condition.

If we can come up with a new API to handle the code lens requirement, we will be able to remove the initial call to the getSyntaxTree API.

However, we still need the syntax tree to work with the data mapper, service designer, and statement editor.