i-net-software / JWebAssembly-Gradle

Gradle plugin for JWebAssembly, a Java to WebAssembly compiler
Apache License 2.0
10 stars 3 forks source link

How are source files chosen to be processed? #2

Closed jmccay-work closed 1 year ago

jmccay-work commented 1 year ago

How are the code files chosen to be processed? I have a war gradle project that has "providedCompile gradleApi()" in the dependencies. It is constantly trying to compile gradle to wasm. This is causing an out of memory error.

> Task :wasm
    compiler: jwebassembly-compiler-qed-0.4.0010.jar
Parsing error with org/apache/groovy/json/internal/LazyMap.class in file:/mnt/c/dev/tool%20versions/gradle/gradle-6.8.2/lib/groovy-all-1.3-2.5.12.jar
java.lang.OutOfMemoryError: GC overhead limit exceeded
    at de.inetsoftware.classparser.ConstantPool.<init>(ConstantPool.java:54)
    at de.inetsoftware.classparser.ClassFile.<init>(ClassFile.java:85)
    at de.inetsoftware.jwebassembly.module.ModuleGenerator.scanLibraries(ModuleGenerator.java:154)
    at de.inetsoftware.jwebassembly.module.ModuleGenerator.<init>(ModuleGenerator.java:116)
    at de.inetsoftware.jwebassembly.JWebAssembly.compile(JWebAssembly.java:364)
    at de.inetsoftware.jwebassembly.JWebAssembly.compileToBinary(JWebAssembly.java:345)
    at de.inetsoftware.jwebassembly.JWebAssembly.compileToBinary(JWebAssembly.java:313)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at de.inetsoftware.jwebassembly.gradle.WasmCompiler.compile(WasmCompiler.java:205)
    at de.inetsoftware.jwebassembly.gradle.WasmTask$1.execute(WasmTask.java:225)
    at org.gradle.api.internal.file.copy.NormalizingCopyActionDecorator.execute(NormalizingCopyActionDecorator.java:59)
    at org.gradle.api.internal.file.copy.DuplicateHandlingCopyActionDecorator.execute(DuplicateHandlingCopyActionDecorator.java:43)
    at org.gradle.api.internal.file.copy.CopyActionExecuter.execute(CopyActionExecuter.java:40)
    at org.gradle.api.tasks.AbstractCopyTask.copy(AbstractCopyTask.java:133)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:104)
    at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:58)
    at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:51)
    at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:29)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$2.run(ExecuteActionsTaskExecuter.java:494)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:75)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:68)
    at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:153)
    at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:68)
Parsing error with org/apache/groovy/json/internal/NumberValue.class in file:/mnt/c/dev/tool%20versions/gradle/gradle-6.8.2/lib/groovy-all-1.3-2.5.12.jar
java.lang.OutOfMemoryError: GC overhead limit exceeded
    at de.inetsoftware.classparser.ConstantPool.<init>(ConstantPool.java:87)
    at de.inetsoftware.classparser.ClassFile.<init>(ClassFile.java:85)
    at de.inetsoftware.jwebassembly.module.ModuleGenerator.scanLibraries(ModuleGenerator.java:154)
    at de.inetsoftware.jwebassembly.module.ModuleGenerator.<init>(ModuleGenerator.java:116)
    at de.inetsoftware.jwebassembly.JWebAssembly.compile(JWebAssembly.java:364)
    at de.inetsoftware.jwebassembly.JWebAssembly.compileToBinary(JWebAssembly.java:345)
    at de.inetsoftware.jwebassembly.JWebAssembly.compileToBinary(JWebAssembly.java:313)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at de.inetsoftware.jwebassembly.gradle.WasmCompiler.compile(WasmCompiler.java:205)
    at de.inetsoftware.jwebassembly.gradle.WasmTask$1.execute(WasmTask.java:225)
    at org.gradle.api.internal.file.copy.NormalizingCopyActionDecorator.execute(NormalizingCopyActionDecorator.java:59)
    at org.gradle.api.internal.file.copy.DuplicateHandlingCopyActionDecorator.execute(DuplicateHandlingCopyActionDecorator.java:43)
    at org.gradle.api.internal.file.copy.CopyActionExecuter.execute(CopyActionExecuter.java:40)
    at org.gradle.api.tasks.AbstractCopyTask.copy(AbstractCopyTask.java:133)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:104)
    at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:58)
    at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:51)
    at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:29)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$2.run(ExecuteActionsTaskExecuter.java:494)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:75)
    at org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:68)
    at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:153)
    at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:68)
Parsing error with org/apache/groovy/json/internal/ReaderCharacterSource.class in file:/mnt/c/dev/tool%20versions/gradle/gradle-6.8.2/lib/groovy-all-1.3-2.5.12.jar
java.lang.OutOfMemoryError: GC overhead limit exceeded
Parsing error with org/apache/groovy/json/internal/ValueContainer.class in file:/mnt/c/dev/tool%20versions/gradle/gradle-6.8.2/lib/groovy-all-1.3-2.5.12.jar

As I am writing this, it is still trying to compile it, and I will have to stop the build manually.

Horcrux7 commented 1 year ago

What you want to do? How look your Gradle script? Which Gradle version do you use? Questions over questions. I'm not a clairvoyant. You need to write many more details if you want help.

jmccay-work commented 1 year ago

Environment:

The versions of JWebAssembly etc. that I am using are just public forks I did to update the build.gradle files. For example, one of the original build copied the build files to a different location. I changed it so it uses a variable that is define in the local system gradle.properties. Since I build using NetBeans on WSL2, I use the directory to copy the built libraries to the Windows maven local repository under the Users directory. I also added the ability to add a separate Gradle / Groovy script file. I use this to add a repository that the company I work for uses without cluttering up the library build. I have started modifying the code in the library to cut back on the objects created, but it is not compiled into any of the libraries. I was trying to avoid delving too deep into the JWebassembly project itself and changing it.

The project currently is a proof of concept project, and it is single java file that grabs an image from a network camera. I was going to expand it if that simple part worked. Unfortunately, I keep getting out of memory errors. It keeps trying to compile other files than the one single Java file.

buildscript {
    repositories {
        mavenLocal()
    }
    dependencies {
        classpath 'de.inetsoftware:jwebassembly-gradle-qed:0.4.0010'
    }
}

plugins {
    id 'java'
    id 'war'
    id 'maven-publish'
}

apply plugin: 'de.inetsoftware.jwebassembly'

// Allow people to developers who build their own to put in a private/company script to
// load the custom maven publishing repository
// Set localRepositoryScript in gradle.properties.
if (project.hasProperty('localRepositoryScript') && project.localRepositoryScript.length() > 0) {
    File repScript = new File("$localRepositoryScript")
    if (repScript.exists()) {
        apply from: "$localRepositoryScript"
    }
}

archivesBaseName = 'videoCore'
String archivePreferredName = null;

//
// Allow a developer building their own version the ability to add their own appendix.
// Set theAppendix in gradle.properties in the root directory.
if (project.hasProperty('theAppendix') && theAppendix.length() > 0) {
    archivePreferredName = String.format("%s-%s", archivesBaseName, "$theAppendix")
} else {
    archivePreferredName = archivesBaseName
}

sourceCompatibility = JavaVersion.VERSION_1_8//1.8
targetCompatibility = JavaVersion.VERSION_1_8//1.8

description = 'Webassembly core of video components.'
group = 'com.qed'
repositories {
    mavenLocal()
    mavenCentral()
}

dependencies {
    // see https://docs.gradle.org/6.8.2/dsl/org.gradle.api.artifacts.dsl.DependencyHandler.html
    providedCompile gradleApi() //api replaces implementation in depencies
    //implementation
    providedCompile 'de.inetsoftware:jwebassembly-api-qed:0.4.0010'
    providedCompile 'de.inetsoftware:jwebassembly-api-qed:0.4.0010:sources'
    providedCompile 'de.inetsoftware:jwebassembly-api-qed:0.4.0010:javadoc'
    providedCompile 'de.inetsoftware:jwebassembly-gradle-qed:0.4.0010'
    providedCompile 'de.inetsoftware:jwebassembly-gradle-qed:0.4.0010:sources'
    providedCompile 'de.inetsoftware:jwebassembly-gradle-qed:0.4.0010:javadoc'
    providedCompile 'de.inetsoftware:jwebassembly-compiler-qed:0.4.0010'
    providedCompile 'de.inetsoftware:jwebassembly-compiler-qed:0.4.0010:sources'
    providedCompile 'de.inetsoftware:jwebassembly-compiler-qed:0.4.0010:javadoc'

    providedCompile 'jakarta.platform:jakarta.jakartaee-web-api:8.0.0'
}

war {
    archiveAppendix.set("$theAppendix")
    archiveBaseName = "videoCore"
    archiveExtension = "war"
    includeEmptyDirs = false    
    manifest {
        attributes(
            "Specification-Title" : "Video Core Components",
            "Specification-Version" : project.version,
            "Specification-Vendor" : "Q.E.D., Inc.",
            "Implementation-Title" : "com.qed.videoCore",
            "Implementation-Version" : project.version,
            "Implementation-Vendor" : "Q.E.D., Inc."
        )
    }
}

// task to list deprecated stuff in the compile
tasks.withType(JavaCompile) {
    options.deprecation = true
    options.compilerArgs += ['-Xlint:deprecation']
    options.compilerArgs += ['-Xlint:unchecked']
}

task sourcesJar(type: Jar) {
   archiveBaseName=archivesBaseName
   archiveAppendix.set("$theAppendix")
   classifier 'sources'
   from sourceSets.main.allSource
}

task javadocJar(type: Jar, dependsOn: javadoc) {
   archiveBaseName=archivesBaseName
   archiveAppendix.set("$theAppendix")
   classifier 'javadoc'
   from javadoc.destinationDir
}

tasks.withType(PublishToMavenRepository) {
    doFirst {
        println("Publishing ${publication.groupId}:${publication.artifactId}:${publication.version} to ${repository.url}")
    }
}

build {
    dependsOn sourcesJar
    dependsOn javadocJar
}

artifacts{
    archives sourcesJar
    archives javadocJar
} 

publishing {
    publications {
        release(MavenPublication) {
            from components.java

            artifactId archivePreferredName

            artifact(tasks.named("sourcesJar").get())
            artifact(tasks.named("javadocJar").get())
        }

    }

    repositories {
        mavenLocal()
    }

}

wasm {
    //Get more debug information if there are problems.
    //logging.level = LogLevel.DEBUG
    //logging.levelInternal = LogLevel.DEBUG

    //Change the output format to the text format *.wat. 
    //This can be helpful to understand the problems if you see the generated code as text format.
    //format = 'text'

    //The used JWebAssembly compiler version. The default is the latest release '+'. 
    //You can set any valid Gradle version string or a full dependency string. 
    //The snapshot version is recommended until a stable release.
    //compilerVersion = 0.2
    //compilerVersion = 'com.github.i-net-software:jwebassembly:master-SNAPSHOT'
    compilerVersion = 'de.inetsoftware:jwebassembly-compiler-qed:0.4.0010'

    //Write method and parameter names into the output *.wasm file. 
    //The file will be approximate 10% larger. And it generate a source map. 
    //With a source map you can see in the debugger of the browser your source code if available.
    //debugNames = true

    //Set an absolute or relative path between the final wasm file location and the source files location.
    ///This is needed for debugging in the browser. It work only if debugNames = true.
    //sourceMapBase = '../../src/main/java/'
    //sourceMapBase = '/src/main/java/'

    //set the destination dir
    //destinationDir = file( 'targetDir' )
    destinationDirectory = file( 'build/wasm' )

    //ignore all referenced native methods without declared replacement in a library and replace them with a stub that throws an exception at runtime
    //ignoreNative = true
}
Horcrux7 commented 1 year ago

Do you want compile the Gradle runtime to Wasm? This will not work because Gradle use for example the file API which can't be used in the browser. The same is valid for the jakarta.jakartaee-web-api. This used Java API which is not availabe in the b browser.

How are the code files chosen to be processed?

The wasm task use the output from the compileJava task. This means you define the code that you compile with the sourceSets. If you does not use the defaults then you define it with:

sourceSets {
    main {
        java {
            srcDir 'yourSrcPath'
        }
    }
}

I have a war gradle project that has "providedCompile gradleApi()"

The only use case for gradleApi() that I know is to write your own Gradle Plugin. I does not understand what you want do with gradleApi() and wasmin one build script.

The next that i does not understand is why you try to use war and a wasm task in one build script. I think both use the same sourceSet. But I have never use it.

If you want build a war file which contains a wasm file then you need two build scripts. First you compile the Java client sources for the wasm file to a single .wasm file. In the other script you build the .war file with the Java server sources for the war file and add *.wasm file as resource file.

jmccay-work commented 1 year ago

I really just want the WASM file. I don't want to compile the Gradle plugin. I just want to compile the one java file.

Does the Gradle plugin have a publication?

Horcrux7 commented 1 year ago

Does the Gradle plugin have a publication?

What you means with it?

jmccay-work commented 1 year ago

Sorry I was talking about a maven publication element. Thanks for you help. Unfortunately, I have to move on to other projects. At this time, it doesn't seem like can be a viable part of the project I am working on right now. I love the work you have done so far. Eventually, I will come back around to playing with this because it shows great potential.

jmccay-work commented 1 year ago

Thanks again.