espertechinc / esper

Esper Complex Event Processing, Streaming SQL and Event Series Analysis
GNU General Public License v2.0
839 stars 259 forks source link

Improve compile efficiency #198

Closed zhucj1981 closed 4 years ago

zhucj1981 commented 4 years ago

In the following code, compile time increase significantly compare with epl statement number.

It cost 162 seconds on my pc.

import java.util.Date;

import com.espertech.esper.common.client.EPCompiled;
import com.espertech.esper.common.client.configuration.Configuration;
import com.espertech.esper.compiler.client.CompilerArguments;
import com.espertech.esper.compiler.client.EPCompiler;
import com.espertech.esper.compiler.client.EPCompilerProvider;

public class Test9 {
    public static void main(String[] args) {
        Configuration configuration = new Configuration();
        CompilerArguments arguments = new CompilerArguments(configuration);
        EPCompiler epCompiler = EPCompilerProvider.getCompiler();
        try {
            Date start = new Date();
            for(int i=0;i<500;i++) {
                EPCompiled compiled = epCompiler.compile(String.format("create variable double V%d=0",i), arguments);
                arguments.getPath().add(compiled);
            }
            Date now = new Date();
            System.out.println("cost "+(now.getTime()-start.getTime())/1000+" seconds");
        } catch (Exception e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }
}
bernhardttom commented 4 years ago

Use the runtime path instead

bernhardttom commented 4 years ago

I see your point now. However what you are doing is equivalent to running "javac" on each file separately. We can improve the API and allow compiling multiple modules at a time.

zhucj1981 commented 4 years ago

In my case, I have lots of epls, and some epl may be depend on others, so i need add compiled to compile path. Because compile process is stateless, each call of compile will re-generate some information from same compiled object in compile path again and again, there is a lot of redundancy.

bernhardttom commented 4 years ago

changes will be in the next release