ballerina-platform / ballerina-lang

The Ballerina Programming Language
https://ballerina.io/
Apache License 2.0
3.68k stars 753 forks source link

[Improvement]: Reduce the LS Package Loader Central Packages Loading time #43172

Open LakshanWeerasinghe opened 4 months ago

LakshanWeerasinghe commented 4 months ago

Description

Currently, the Language Server package loader is responsible for loading packages under the ballerinax organization using the Central API client. This approach loads information about only 10 packages at a time, which significantly increases the time taken to complete the LS package loading process, ultimately resulting in a poor developer experience (DX).

Describe your problem(s)

No response

Describe your solution(s)

To improve this DX and reduce the central package loading time, we have identified three potential approaches:

Advantages: Faster than fetching packages individually since the information is pre-computed. The JSON file size can be optimized by removing unnecessary attributes, reducing the data transfer load.

Disadvantages: Users may not receive the most up-to-date information about available packages, as the data is only updated once daily.

Advantages: Easy to maintain as it leverages an existing API. Ensures users receive up-to-date package information.

Disadvantages: Relatively slower compared to the first approach due to the performance of the central API.

Advantages:

Reduces the overall loading time significantly. Disadvantages: Relatively slower compared to the first approach due to the performance of the central API.

Latency comparison between REST and GraphQL apis haven't been done. Since we get more precise and compressed information though the GraphQL API it should be faster.

Related area

-> Compilation

Related issue(s) (optional)

No response

Suggested label(s) (optional)

No response

Suggested assignee(s) (optional)

No response

LakshanWeerasinghe commented 3 months ago

After the discussion with the team members we decided to do two following things.

  1. To provide the import module completions of packages in the ballerina and ballerinax orgs use the GraphQL API provided by the Ballerina Central.
    
    https://api.central.ballerina.io/2.0/graphql

query Packages { packages(orgName: "ballerina", limit: 1000) { packages { name version } } }


2. Create a index to which contains the information about the available Listeners in packages in the `ballerina` and `ballerinax` orgs. Use this index to populate the Service Templates Snippets. This  index will be packed with the Ballerina Distribution and also it will be available in the Ballerina Central as well. The index in the Central will be updated daily/on-demand.
LakshanWeerasinghe commented 3 months ago
LakshanWeerasinghe commented 2 months ago
  1. Store LS Indexes
    isolated resource function post ls\-index(http:Request req)
    returns http:InternalServerError|http:BadRequest|http:Created;
  2. Retrieve LS Indexes
    isolated resource function get ls\-index(string bal_version) 
    returns http:InternalServerError|http:BadRequest|http:Response;
  3. Get LS Index checksum for specific Ballerina version
    isolated resource function get ls\-index/checksum(string bal_version) 
    returns json|http:InternalServerError|http:BadRequest;

High-level architecture view

image