support incremental builds

[domenique]

It seems that this plugin is always regenerating java objects. it would be nice if it supported incremental builds:

https://docs.gradle.org/current/userguide/custom_tasks.html#incremental_tasks

Any chance of implementing this feature ? it would greatly improve our build time!

[ciscoo]

Hmm that's interesting. My understanding was that it was already incremental since the built-in JavaExec is used.

Looking at Gradle's tests, they use the file operation instead of dir like this plugin does:

• https://github.com/gradle/gradle/blob/v6.8.2/subprojects/plugins/src/integTest/groovy/org/gradle/api/tasks/JavaExecIntegrationTest.groovy#L107..L134
• https://github.com/ciscoo/cxf-codegen-gradle/blob/v1.0.0-rc.1/cxf-codegen-gradle/src/main/java/io/mateo/cxf/codegen/CxfCodegenPlugin.java#L99

So I wonder if file is only supported for incremental builds or I'm missing something here.

Nonetheless, I'll look into it. Happy to see someone using the plugin 😄

[domenique]

Hi @ciscoo,

I've done some more research and it seems that the plugin should be doing it correctly... only I have a wsdl which includes references to other XSD's, and the change detection says the outputs have changed because of the types in that common XSD :(

when doing gradle wsdl2java --info I get the following output:

Task ':xxxx:wsdl2javaXxxServiceV1' is not up-to-date because:
  Output property '$1' file /xxx/src/main/generated-cxf/be/xx/xx/types/common/v3/BaseRequestType.java has changed.
  Output property '$1' file /xxx/src/main/generated-cxf/be/xx/xx/types/common/v3/BaseResponseType.java has changed.
  Output property '$1' file /xxx/src/main/generated-cxf/be/xx/xxx/types/xxx/v1/DataGroupElementType.java has changed.

could it be related to the fact that the common types are declared in a seperate XSD and that the plugin builds those first before the change detection kicks in?

[ciscoo]

could it be related to the fact that the common types are declared in a seperate XSD

That doesn't appear to affect incremental builds. I added explicit tests to be certain for both a single WSDL and one where an XSD is imported, see https://github.com/ciscoo/cxf-codegen-gradle/commit/b028adaa746a35694723aa211a1d44ed8edb1925.

This confirms my understanding that as is, the Wsdl2JavaTask is incremental by extension of the JavaExec task.

I see from the output you provided that you've changed the output to src/main. You may have other plugins or something else at play here that is changing the generated files somehow. By default, the output directory is the build/ directory which is typically ignored.

However, some plugins, such as Spotless, will format Java files that are part of the main source set, which means you'll need to explicitly ignore the generated Java sources otherwise Spotless will format them which will cause the outputs to be changed.

[domenique]

Hi @ciscoo

I think I've found a possible reason... I'm using common XSD files for a bunch of common types, I declared 4 wsdl in the config file, and they all use the same common types.

The second WSDL overwrites the common types of the first one... marking them as updated for the next run I assume.

Is there a way to process multiple WSDL files in one go instead of a run for each wsdl ?

[ciscoo]

Ahh okay that makes sense as to why Gradle detected that the outputs were out of date.

Is there a way to process multiple WSDL files in one go instead of a run for each wsdl ?

This plugin is limited to what the underlying wsdl2java tool can do. Working around those limitations is not something in scope of this plugin.

Based on what I see here:

• https://cxf.apache.org/docs/wsdl-to-java.html
• https://github.com/apache/cxf/blob/cxf-3.4.2/tools/wsdlto/frontend/jaxws/src/main/java/org/apache/cxf/tools/wsdlto/frontend/jaxws/jaxws-toolspec.xml#L416..L424

The wsdlList configuration may be what you're looking for. However, looking at the underlying implementation as to what the tool is doing, I see these lines. So my understanding of those lines, the tool will essentially loop over each line and generate Java sources which leads us back to where we are now: out of date builds.

~Looking at this plugins implementation of wsdlList, it isn't right since you can't configure a wsdlUrl property for the tool to use -- todo for me.~ 2020-02-10 Edit: Maven cxf-codegen-plugin does not have a wsdlUrl command argument as indicated by the generated command that is passed to the wsdl2java tool. So as a result, this plugin also does not have a wsdlUrl property either.

[ciscoo]

Closing since there is nothing to do from the plugin's perspective.

[Remboooo]

I do want to chime in here since I'm facing the same issue as OP.

@ciscoo

The wsdlList configuration may be what you're looking for. However, looking at the underlying implementation as to what the tool is doing, I see these lines. So my understanding of those lines, the tool will essentially loop over each line and generate Java sources which leads us back to where we are now: out of date builds.

I don't see how wsdlList property wouldn't solve this issue. The problem is that we now have to define separate tasks that interfere with each other's outputs in the case of common included XSDs. You can't solve this by defining separate output directories, because that would result in duplicate conflicting class definitions. By putting all wsdl2java invocations into 1 task (using a wsdlList property), the combined outputs of the invocations are considered as 1 set of outputs of 1 task. The problem of multiple tasks touching each other's outputs is then eliminated, right?

[ciscoo]

I was taking a guess by walking through the code paths by eyeball. I don't have any example WSDLs/XSDs that share common items to confirm.

Everything you have said makes sense to me. The only way to verify that is to test it out and see what Gradle reports back when executing the task with --info log level enabled. Are you able to verify with the WSDLs and common XSDs you have?

[Remboooo]

I have tried a run with --info and I interpret the output as confirming that the tasks overwriting each other's outputs is the issue:

> Task :wsdl2java-dsgld
Caching disabled for task ':wsdl2java-dsgld' because:
  Build cache is disabled
Task ':wsdl2java-dsgld' is not up-to-date because:
  Output property 'wsdl2JavaOptions.outputDir' file C:\Users\RembrandvanLakwijk\dev\backend\build\generated-src\bro\net\opengis\gml\_3\AbstractFeatureType.java has changed.
  Output property 'wsdl2JavaOptions.outputDir' file C:\Users\RembrandvanLakwijk\dev\backend\build\generated-src\bro\net\opengis\gml\_3\CodeWithAuthorityType.java has changed.
  Output property 'wsdl2JavaOptions.outputDir' file C:\Users\RembrandvanLakwijk\dev\backend\build\generated-src\bro\net\opengis\gml\_3\FeaturePropertyType.java has changed.
Starting process 'command 'C:\Program Files\Eclipse Adoptium\jdk-17.0.3.7-hotspot\bin\java.exe''. Working directory: C:\Users\RembrandvanLakwijk\dev\backend Command: C:\Program Files\Eclipse Adoptium\jdk-17.0.3.7-hotspot\bin\java.exe -Dfile.encoding=UTF-8 -Duser.country=NL -Duser.language=nl -Duser.variant -cp ... classpath ... org.apache.cxf.tools.wsdlto.WSDLToJava -encoding UTF-8 -d C:\Users\RembrandvanLakwijk\dev\backend\build\generated-src\bro -b file:/C:/Users/RembrandvanLakwijk/dev/backend/src/main/resources/bro/schema/dsgld-bindings.xjb -catalog C:\Users\RembrandvanLakwijk\dev\backend/src/main/resources/bro/schema/catalog.xml -autoNameResolution file:/C:/Users/RembrandvanLakwijk/dev/backend/src/main/resources/bro/schema/dsgld-v1.0.wsdl
Successfully started process 'command 'C:\Program Files\Eclipse Adoptium\jdk-17.0.3.7-hotspot\bin\java.exe''
Resolve mutations for :wsdl2java-dsgmn (Thread[Execution worker,5,main]) started.
:wsdl2java-dsgmn (Thread[Execution worker,5,main]) started.

> Task :wsdl2java-dsgmn
Caching disabled for task ':wsdl2java-dsgmn' because:
  Build cache is disabled
Task ':wsdl2java-dsgmn' is not up-to-date because:
  Output property 'wsdl2JavaOptions.outputDir' file C:\Users\RembrandvanLakwijk\dev\backend\build\generated-src\bro\net\opengis\gml\_3\AbstractFeatureType.java has changed.
  Output property 'wsdl2JavaOptions.outputDir' file C:\Users\RembrandvanLakwijk\dev\backend\build\generated-src\bro\net\opengis\gml\_3\CodeWithAuthorityType.java has changed.
  Output property 'wsdl2JavaOptions.outputDir' file C:\Users\RembrandvanLakwijk\dev\backend\build\generated-src\bro\net\opengis\gml\_3\FeaturePropertyType.java has changed.
Starting process 'command 'C:\Program Files\Eclipse Adoptium\jdk-17.0.3.7-hotspot\bin\java.exe''. Working directory: C:\Users\RembrandvanLakwijk\dev\backend Command: C:\Program Files\Eclipse Adoptium\jdk-17.0.3.7-hotspot\bin\java.exe -Dfile.encoding=UTF-8 -Duser.country=NL -Duser.language=nl -Duser.variant -cp ... classpath ... org.apache.cxf.tools.wsdlto.WSDLToJava -encoding UTF-8 -d C:\Users\RembrandvanLakwijk\dev\backend\build\generated-src\bro -b file:/C:/Users/RembrandvanLakwijk/dev/backend/src/main/resources/bro/schema/dsgmn-bindings.xjb -catalog C:\Users\RembrandvanLakwijk\dev\backend/src/main/resources/bro/schema/catalog.xml -autoNameResolution file:/C:/Users/RembrandvanLakwijk/dev/backend/src/main/resources/bro/schema/dsgmn-v1.0.wsdl
Successfully started process 'command 'C:\Program Files\Eclipse Adoptium\jdk-17.0.3.7-hotspot\bin\java.exe''
Resolve mutations for :wsdl2java-dsgmw (Thread[Execution worker,5,main]) started.
:wsdl2java-dsgmw (Thread[Execution worker,5,main]) started.

I can probably cobble up a minimum working example with these WSDLs and XSDs if that helps?

[ciscoo]

Yes please

[Remboooo]

I've created a minimal example here: https://github.com/Remboooo/cxf-codegen-overlap-test If you run :wsdl2java twice, it will run the tasks each time stating:

  Output property 'wsdl2JavaOptions.outputDir' file C:\Users\RembrandvanLakwijk\dev\cxf-codegen-example\build\generated-src\bro\net\opengis\gml\_3\AbstractFeatureType.java has changed.
  Output property 'wsdl2JavaOptions.outputDir' file C:\Users\RembrandvanLakwijk\dev\cxf-codegen-example\build\generated-src\bro\net\opengis\gml\_3\CodeWithAuthorityType.java has changed.
  Output property 'wsdl2JavaOptions.outputDir' file C:\Users\RembrandvanLakwijk\dev\cxf-codegen-example\build\generated-src\bro\net\opengis\gml\_3\FeaturePropertyType.java has changed.

[Remboooo]

In the mean time, I found a workaround. I finally understand now that the -wsdlList argument to CXF means that the supplied wsdlUrl should point to a plain text file with 1 WSDL path or URL per line. With that knowledge, it is possible to have the cxf-codegen-gradle plugin generate all sources in one task, eliminating the problem of the task always being out-of-date:

def wsdlListFile = new File("${buildDir}/wsdlList.tmp")

tasks.register('generateWsdlList') {
    group = "wsdl2java"
    doFirst({
        wsdlListFile.text = [
                "/path/to/wsdl/1.wsdl",
                "/path/to/wsdl/2.wsdl"
        ].join("\n")
    })
    outputs.file(wsdlListFile)
}

tasks.register("myWsdl2javaTask", Wsdl2Java) {
    dependsOn 'generateWsdlList'
    inputs.files(fileTree("/path/to/wsdl"))
    toolOptions {
        wsdl.set(wsdlListFile)
        wsdlList.set(true)
        bindingFiles.add("/path/to/wsdl/bindings.xjb")
        catalog.set("/path/to/wsdl/catalog.xml")
        encoding.set("UTF-8")
        autoNameResolution.set(true)
        outputDir.set(file("$generatedSourcesDir/wsdl2java-bindings"))
    }
}

[Fafan2008]

@Remboooo You are a genius !!! Thanks a lot for the research and the answer! @ciscoo Thank you for this plugin)