Can't compile if the file content is too large

[Rajith90]

Description: I have a large json structure which is an open API definition. When build my project I get the following error

`Compiling source wso2/rajiNew:0.1.0

Creating balos target/balo/rajiNew-2019r3-any-0.1.0.balo error: wso2:rajiNew:rajiNew:1:1: method is too large: 'wso2/rajiNew:0.1.0.'`

Steps to reproduce: Please find the file attached here. openAPIJsonConstants.bal.zip Affected Versions: 1.0.0

OS, DB, other environment details and versions:

Related Issues (optional):

Suggested Labels (optional):

Suggested Assignees (optional):

[SupunS]

As per the JVM spec (https://docs.oracle.com/javase/specs/jvms/se8/html/index.html) the maximum size of a method is 65535 bytes. Since jBallerina compiles down to java-byte-code and runs on top of JVM above method size limit also applies to ballerina methods (only for jballerina. This will not be applicable to other implementations such as native implementations - i.e: nballerina).

However, since how jballerina generates byte code is different to how java generates the byte code, this method size limit is different for java method and ballerina methods. With future byte-code optimizations, this limit will be increased, however, it is not possible to completely remove it as its enforced by the underlying JVM.

The solution is:

• For large JSON/XML - Read it from a file - this is anyway the recommended way to read large data files such as large JSON/XML/etc. If you need to avoid IO operation, then store the JSON as a string and parse it to JSON at runtime.
• For large methods - split the methods into much more compact methods.

[praminda]

We need to read a json/yaml file of unknown size (can be 1-5MB) for a feature implementation in wso2 microgateway. However right now with jballerina (1.1.x) it looks like its not packing project resources to the final jar file. Which makes it harder for us to implement read from file solution. Is there any other workarounds or ways to pack resource files to the ballerina jar file.

[riyafa]

The method size allowed will be increased to a great extent with the local variable optimization done using liveness analysis in the PR: https://github.com/ballerina-platform/ballerina-lang/pull/25049

[chanukaranaba]

The following code will help to reproduce this issue. This is giving the **Error: Method is too large

function insertOpportunities(sfdc:SoqlResult opportunities) returns int|error{
    //io:println(opportunities); 
    io:println("Start inserting Opportunities..."); 
    azureMysql:Client dbClient = check createAzureMySQLClient();

    sql:ParameterizedQuery updateQuery = `UPDATE chanuka_test SET IsInSF = 0`;
    sql:ExecutionResult result = check dbClient->execute(updateQuery);
    io:println("IsInSF Updated Row count: ", result?.affectedRowCount);

    sql:ParameterizedQuery[] insertQueries = [];
    int i = 0;

    // Creates a batch-parameterized query.
    foreach var opportunity in opportunities.records {
        string Id = opportunity["Id"].toString();
        string Name = opportunity["Name"].toString();
        string AccountId = opportunity["AccountId"].toString();
        string BANT_Chanel__c = opportunity["BANT_Chanel__c"].toString();
        string Closed_QTR__c = opportunity["Closed_QTR__c"].toString();
        string CurrencyIsoCode = opportunity["CurrencyIsoCode"].toString();
        string Development_Support_Account_Key__c = opportunity["Development_Support_Account_Key__c"].toString();
        string Engagement_Code__c = opportunity["Engagement_Code__c"].toString();

        string Account_Address__c = opportunity["Account_Address__c"].toString();
        string Pre_Sales_Code__c = opportunity["Pre_Sales_Code__c"].toString();
        string OwnerId = opportunity["OwnerId"].toString();
        string Owner_ID__c = opportunity["Owner_ID__c"].toString();
        string Technical_Owner__c = opportunity["Technical_Owner__c"].toString();
        string Region__c = opportunity["Region__c"].toString();
        string Entry_Vector__c = opportunity["Entry_Vector__c"].toString();
        string StageName = opportunity["StageName"].toString();
        string Account_Classification__c = opportunity["Account_Classification__c"].toString();
        string Account_Sub_Category__c = opportunity["Account_Sub_Category__c"].toString();

        int IsClosed = convertBooleanToBit(<boolean>opportunity["IsClosed"]);
        int IsWon = convertBooleanToBit(<boolean>opportunity["IsWon"]);
        int Reached_Proposal_Stage__c = convertBooleanToBit(<boolean>opportunity["Reached_Proposal_Stage__c"]);
        int IsInSF = 1;

        string? CL_End_Date_Roll_Up__c = dateConvert(opportunity["CL_End_Date_Roll_Up__c"]);
        string? BANT_Qualified_First_Sale_Date__c = dateConvert(opportunity["BANT_Qualified_First_Sale_Date__c"]);
        string? CloseDate = dateConvert(opportunity["CloseDate"]);
        string? Opp_End_Date__c = dateConvert(opportunity["Opp_End_Date__c"]);
        string? Opp_Start_Date__c = dateConvert(opportunity["Opp_Start_Date__c"]);
        string? CreatedDate = dateConvert(opportunity["CreatedDate"]);
        string? PS_Support_Account_End_Date_Roll_Up__c = dateConvert(opportunity["PS_Support_Account_End_Date_Roll_Up__c"]);
        string? PS_Support_Account_Start_Date_Roll_Up__c = dateConvert(opportunity["PS_Support_Account_Start_Date_Roll_Up__c"]);
        string? Proposal_Stage_Date__c = dateConvert(opportunity["Proposal_Stage_Date__c"]);

        decimal ARR__C =  <decimal>opportunity["ARR__c"];
        decimal Amount =  <decimal>opportunity["Amount"];
        decimal Closed_Month__c =  <decimal>opportunity["Closed_Month__c"];
        decimal Development_Support_Hours__c =  <decimal>opportunity["Development_Support_Hours__c"];

       insertQueries[i] =  `INSERT INTO chanuka_test
                  (Id, Name, Amount,ARR__C, CL_End_Date_Roll_Up__c, AccountId, BANT_Chanel__c, BANT_Qualified_First_Sale_Date__c, CloseDate, Closed_Month__c, Closed_QTR__c,
                  CurrencyIsoCode, Development_Support_Account_Key__c, Development_Support_Hours__c,Engagement_Code__c,IsClosed,IsWon,Opp_End_Date__c,Opp_Start_Date__c,CreatedDate,
                  Account_Address__c,Pre_Sales_Code__c,OwnerId,Owner_ID__c,Technical_Owner__c,Region__c,Entry_Vector__c,StageName,PS_Support_Account_End_Date_Roll_Up__c,
                  PS_Support_Account_Start_Date_Roll_Up__c,Account_Classification__c,Reached_Proposal_Stage__c,Account_Sub_Category__c,Proposal_Stage_Date__c,IsInSF)
                  VALUES (${Id}, ${Name},${Amount},${ARR__C},${CL_End_Date_Roll_Up__c},${AccountId},${BANT_Chanel__c},${BANT_Qualified_First_Sale_Date__c},
                  ${CloseDate},${Closed_Month__c},${Closed_QTR__c},${CurrencyIsoCode},${Development_Support_Account_Key__c},${Development_Support_Hours__c},${Engagement_Code__c},${IsClosed},${IsWon},
                  ${Opp_End_Date__c},${Opp_Start_Date__c},${CreatedDate},${Account_Address__c},${Pre_Sales_Code__c},${OwnerId},${Owner_ID__c},${Technical_Owner__c},${Region__c},${Entry_Vector__c},${StageName},${PS_Support_Account_End_Date_Roll_Up__c},${PS_Support_Account_Start_Date_Roll_Up__c},${Account_Classification__c},
                  ${Reached_Proposal_Stage__c},${Account_Sub_Category__c},${Proposal_Stage_Date__c},${IsInSF}) ON DUPLICATE KEY UPDATE 
                  Name = ${Name}, Amount = ${Amount},ARR__C = ${ARR__C},CL_End_Date_Roll_Up__c = ${CL_End_Date_Roll_Up__c}, AccountId = ${AccountId},
                  BANT_Chanel__c = ${BANT_Chanel__c}, BANT_Qualified_First_Sale_Date__c = ${BANT_Qualified_First_Sale_Date__c},CloseDate = ${CloseDate}, Closed_Month__c = ${Closed_Month__c},
                  Closed_QTR__c = ${Closed_QTR__c}, CurrencyIsoCode = ${CurrencyIsoCode},Development_Support_Account_Key__c = ${Development_Support_Account_Key__c}, 
                  Development_Support_Hours__c = ${Development_Support_Hours__c},Engagement_Code__c = ${Engagement_Code__c}, IsClosed = ${IsClosed},IsWon = ${IsWon},Opp_End_Date__c = ${Opp_End_Date__c},Opp_Start_Date__c = ${Opp_Start_Date__c},
                  CreatedDate = ${CreatedDate},Account_Address__c = ${Account_Address__c},Pre_Sales_Code__c = ${Pre_Sales_Code__c},OwnerId = ${OwnerId},Owner_ID__c = ${Owner_ID__c},Technical_Owner__c = ${Technical_Owner__c},Region__c = ${Region__c},Entry_Vector__c = ${Entry_Vector__c},
                  StageName = ${StageName},PS_Support_Account_End_Date_Roll_Up__c = ${PS_Support_Account_End_Date_Roll_Up__c},PS_Support_Account_Start_Date_Roll_Up__c = ${PS_Support_Account_Start_Date_Roll_Up__c},Account_Classification__c = ${Account_Classification__c},Reached_Proposal_Stage__c = ${Reached_Proposal_Stage__c},
                  Account_Sub_Category__c = ${Account_Sub_Category__c},Proposal_Stage_Date__c = ${Proposal_Stage_Date__c},IsInSF = ${IsInSF}`;

        i = i + 1;
    }

    // Inserts the records with the auto-generated ID.
    sql:ExecutionResult[] resultInsert = check dbClient->batchExecute(insertQueries);
    io:println(resultInsert);
    int numberOfRowsInserted = 0;
    foreach var summary in resultInsert {
        numberOfRowsInserted = numberOfRowsInserted + <int> summary.affectedRowCount;
    }
    io:println ("Number of rows inserted: " + numberOfRowsInserted.toString());
    return numberOfRowsInserted;
}

[warunalakshitha]

This issue happen due to sql:ParameterizedQuery which is using raw templates. Internally raw templates use type alias with union types. public type Value string|int|boolean|float|decimal|byte[]|xml|DateTimeType|ArrayValueType|TypedValue?;

For each template value runtime creates union of above types which generates consider amount of code. But for each value we should be able to reuse same union type. ATM those type alias names are not passed to codegen from compiler Front end due to https://github.com/ballerina-platform/ballerina-lang/issues/26284.

Fixing https://github.com/ballerina-platform/ballerina-lang/issues/26284 will solve above issue.

We also working on optimizing union type ( anonymous) creation logic as well. https://github.com/ballerina-platform/ballerina-lang/issues/32019