mojohaus / jaxb2-maven-plugin

JAXB2 Maven Plugin
https://www.mojohaus.org/jaxb2-maven-plugin/
Apache License 2.0
106 stars 77 forks source link

Schemagen does not handle paths with spaces correctly #48

Closed rwm closed 6 years ago

rwm commented 8 years ago

A very simple schemagen invokation with maven will fail if the path contains spaces. Tested with jaxb-maven-plugin 2.2 on Windows 7. My guess is that somewhere paths are urlencoded but not treated as urls. The exception is caused by an illegal path to the file sun-jaxb.episode.

If I run the same command with exactly the same files from a directory without spaces, no error occurs.

Complete error message below:

$ mvn clean install
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building AKTIN : DWH API 0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ dwh-api ---
[INFO] Deleting C:\temp\test\dwh api\target
[INFO]
[INFO] --- jaxb2-maven-plugin:2.2:schemagen (schemagen) @ dwh-api ---
[ERROR] Execution failed.
[ERROR]
[Exception]: java.io.FileNotFoundException
[Message]: C:\temp\test\dwh%20api\target\generated-resources\schemagen\META-INF\sun-jaxb.episode (Das System kann den angegebenen Pfad nicht finden)
         java.io.FileOutputStream.open(Native Method)
         java.io.FileOutputStream.<init>(FileOutputStream.java:213)
         java.io.FileOutputStream.<init>(FileOutputStream.java:101)
         com.sun.xml.txw2.output.StreamSerializer.<init>(StreamSerializer.java:92)
         com.sun.xml.txw2.output.ResultFactory.createSerializer(ResultFactory.java:75)
         com.sun.tools.jxc.api.impl.j2s.JAXBModelImpl.generateEpisodeFile(JAXBModelImpl.java:187)
         com.sun.tools.jxc.ap.SchemaGenerator.process(SchemaGenerator.java:141)
         com.sun.tools.javac.processing.JavacProcessingEnvironment.callProcessor(JavacProcessingEnvironment.java:794)
         com.sun.tools.javac.processing.JavacProcessingEnvironment.discoverAndRunProcs(JavacProcessingEnvironment.java:705)
         com.sun.tools.javac.processing.JavacProcessingEnvironment.access$1800(JavacProcessingEnvironment.java:91)
         com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.run(JavacProcessingEnvironment.java:1035)
         com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing(JavacProcessingEnvironment.java:1176)
         com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler.java:1170)
         com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:856)
         com.sun.tools.javac.main.Main.compile(Main.java:523)
         com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129)
         com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138)
         com.sun.tools.jxc.SchemaGenerator$Runner.compile(SchemaGenerator.java:261)
         sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
         sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
         java.lang.reflect.Method.invoke(Method.java:483)
         com.sun.tools.jxc.SchemaGenerator.run(SchemaGenerator.java:166)
         org.codehaus.mojo.jaxb2.schemageneration.AbstractXsdGeneratorMojo.performExecution(AbstractXsdGeneratorMojo.java:370)
         org.codehaus.mojo.jaxb2.AbstractJaxbMojo.execute(AbstractJaxbMojo.java:257)
         org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:132)
         org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
         org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
         org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
         org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
         org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
         org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
         org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:120)
         org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:347)
         org.apache.maven.DefaultMaven.execute(DefaultMaven.java:154)
         org.apache.maven.cli.MavenCli.execute(MavenCli.java:582)
         org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214)
         org.apache.maven.cli.MavenCli.main(MavenCli.java:158)
         sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
         sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
         java.lang.reflect.Method.invoke(Method.java:483)
         org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
         org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
         org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
         org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.716 s
[INFO] Finished at: 2016-04-27T16:41:49+02:00
[INFO] Final Memory: 17M/210M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:jaxb2-maven-plugin:2.2:schemagen (schemagen) on project dwh-api:
[ERROR] +=================== [SchemaGenerator Error '<unknown>']
[ERROR] |
[ERROR] | SchemaGen did not complete its operation correctly.
[ERROR] |
[ERROR] | To re-create the error (and get a proper error message), cd to:
[ERROR] | C:\temp\test\dwh api
[ERROR] | ... and fire the following on a command line/in a shell:
[ERROR] |
[ERROR] | schemagen -encoding Cp1252 -d C:\temp\test\dwh api\target\schemagen-work\compile_scope -classpath /C:/temp/test/dwh%20api/src/main/java/ -episode C:\temp\test\dwh api\target\generated-resources\schemagen\META-INF\sun-jaxb.episode src\main\java\org\aktin\dwh\Authenticator.java src\main\java\org\aktin\dwh\PreferenceKeys.java src\main\java\org\aktin\exchange\AbstractConcept.java src\main\java\org\aktin\exchange\Principal.java src\main\java\org\aktin\exchange\Query.java src\main\java\org\aktin\exchange\RawConcept.java src\main\java\org\aktin\exchange\XMLConstants.java src\main\java\org\aktin\exchange\package-info.java src\main\java\org\aktin\prefs\Preference.java src\main\java\org\aktin\prefs\Preferences.java src\main\java\org\aktin\report\Report.java
[ERROR] |
[ERROR] | The following source files should be processed by schemagen:
[ERROR] | 0: file:/C:/temp/test/dwh%20api/src/main/java/org/aktin/dwh/Authenticator.java
[ERROR] | 1: file:/C:/temp/test/dwh%20api/src/main/java/org/aktin/dwh/PreferenceKeys.java
[ERROR] | 2: file:/C:/temp/test/dwh%20api/src/main/java/org/aktin/exchange/AbstractConcept.java
[ERROR] | 3: file:/C:/temp/test/dwh%20api/src/main/java/org/aktin/exchange/Principal.java
[ERROR] | 4: file:/C:/temp/test/dwh%20api/src/main/java/org/aktin/exchange/Query.java
[ERROR] | 5: file:/C:/temp/test/dwh%20api/src/main/java/org/aktin/exchange/RawConcept.java
[ERROR] | 6: file:/C:/temp/test/dwh%20api/src/main/java/org/aktin/exchange/XMLConstants.java
[ERROR] | 7: file:/C:/temp/test/dwh%20api/src/main/java/org/aktin/exchange/package-info.java
[ERROR] | 8: file:/C:/temp/test/dwh%20api/src/main/java/org/aktin/prefs/Preference.java
[ERROR] | 9: file:/C:/temp/test/dwh%20api/src/main/java/org/aktin/prefs/Preferences.java
[ERROR] | 10: file:/C:/temp/test/dwh%20api/src/main/java/org/aktin/report/Report.java
[ERROR] |
[ERROR] +=================== [End SchemaGenerator Error]: C:\temp\test\dwh%20api\target\generated-resources\schemagen\META-INF\sun-jaxb.episode (Das System kann den angegebenen Pfad nicht finden)
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

POM file:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <name>AKTIN : DWH API</name>

    <groupId>org.aktin</groupId>
    <artifactId>dwh-api</artifactId>
    <version>0.1-SNAPSHOT</version>

    <build>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>jaxb2-maven-plugin</artifactId>
                <version>2.2</version>
                <executions>
                    <execution>
                        <id>schemagen</id>
                        <goals>
                            <goal>schemagen</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>

    </build>
</project>
lennartj commented 8 years ago

It is recommended to always use paths without whitespace; many tools in the Maven toolchain will encounter problems when trying to encode/decode whitespace properly.

This error stems from the java.net.URL class which replaces whitespace in a path with %20 when doing the following operation:

getClass().getClassLoader().getResource("testdata/shared/dirwith space/somefile.txt").getPath();
lennartj commented 8 years ago

In the 2.3 release, I'll simply update the documentation to indicate that whitespace in project paths is a recipe for somewhat problematic situations.

rwm commented 8 years ago

Thanks for the feedback, adding documentation is a good first measure.

The problem will occur not only with spaces. Any character which will be escaped by URL will cause problems. So basically I'm forced to only use [A-Za-z0-9-_~\.] (regex). Anything else will not work, e.g. + or , or ( will fail. See https://en.wikipedia.org/wiki/Percent-encoding

Up to this point I didn't have any problems with the maven toolchain with my paths containing spaces, so I would definitely call this a bug.

The problem is caused by conversion from URL to File, which should not be done. A safe way would be to keep the URL objects and only access the content with openConnection().

kevinmangold commented 8 years ago

"Up to this point I didn't have any problems with the maven toolchain with my paths containing spaces, so I would definitely call this a bug." -- Agreed -- I've never had an issue with spaces in a path until now -- this is most definitely a bug.

sermojohn commented 8 years ago

We hit this issue as well. I believe its not easy to enforce the non-space directory constraint as a project might be built by a CI server whose directory might be out of the developers' control. So, I agree that this issue should be fixed!

lennartj commented 8 years ago

The problem occurs when going from URL Resources, found by the ClassLoader's getResource(), and converting the resources to Files. This conversion is needed - and hence need to stay in the code - for some reasons:

  1. We need to get files, because the xjc and schemagen tools requires file paths (which are extracted from URLs) to work.
  2. We also need a means to find out which files to include in the JAXB compilation and which to exclude, such as when including half of the source files within a maven project.
  3. Since developers mostly decide which files to include based on the paths to the source files, we need to convert the URLs to File paths to simplify configuration of the plugin for devs.

We do need to get this conversion correct, but the reason it even becomes a problem is that the XJC and SchemaGen tools require file paths as arguments to work. XJC has an option to use URLs but SchemaGen does not.

lennartj commented 8 years ago

This seems to be a problem not related to the Plugin code, but rather to the SchemaGen internal implementation. Having created an IT to validate that the plugin provides non-URLEncoded arguments to the underlying implementation, the arguments are extracted correctly as shown in the printout below (in particular arguments with index 3,5,7 and 8).

[DEBUG] SourceCodeFile [/Users/lj/Development/Projects/Mojohaus/jaxb2-maven-plugin/target/it/schemagen-handles-spaces in paths/src/main/java/se/west/SomeType.java] and userDir [/Users/lj/Development/Projects/Mojohaus/jaxb2-maven-plugin/target/it/schemagen-handles-spaces in paths] ==> relativePath: src/main/java/se/west/SomeType.java. (baseDir: /Users/lj/Development/Projects/Mojohaus/jaxb2-maven-plugin/target/it/schemagen-handles-spaces in paths]
[DEBUG] [ClassName-2-SourcePath Map (size: 1)] ...
[DEBUG]   1/1: [se.west.SomeType]: src/main/java/se/west/SomeType.java
[DEBUG] ... End [ClassName-2-SourcePath Map]
[DEBUG] 
+=================== [9 SchemaGen Arguments]
|
| [0]: -encoding
| [1]: UTF-8
| [2]: -d
| [3]: /Users/lj/Development/Projects/Mojohaus/jaxb2-maven-plugin/target/it/schemagen-handles-spaces in paths/target/schemagen-work/compile_scope
| [4]: -classpath
| [5]: /Users/lj/Development/Projects/Mojohaus/jaxb2-maven-plugin/target/it/schemagen-handles-spaces in paths/src/main/java/
| [6]: -episode
| [7]: /Users/lj/Development/Projects/Mojohaus/jaxb2-maven-plugin/target/it/schemagen-handles-spaces in paths/target/generated-resources/schemagen/META-INF/sun-jaxb.episode
| [8]: src/main/java/se/west/SomeType.java
|
+=================== [End 9 SchemaGen Arguments]

[DEBUG] Created episode directory [/Users/lj/Development/Projects/Mojohaus/jaxb2-maven-plugin/target/it/schemagen-handles-spaces in paths/target/generated-resources/schemagen/META-INF]: true

However, executing the SchemaGen compiler with these arguments, it generates an exception with a stack trace. Hence, it seems that SchemaGen performs URLEncoding internally. In that case, I wonder if SchemaGen has ever worked with whitespace in any of its argument paths?

Caused by: java.io.FileNotFoundException: /Users/lj/Development/Projects/Mojohaus/jaxb2-maven-plugin/target/it/schemagen-handles-spaces%20in%20paths/target/generated-resources/schemagen/META-INF/sun-jaxb.episode (No such file or directory)
    at java.io.FileOutputStream.open0(Native Method)
    at java.io.FileOutputStream.open(FileOutputStream.java:270)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:101)
    at com.sun.xml.txw2.output.StreamSerializer.<init>(StreamSerializer.java:92)
    at com.sun.xml.txw2.output.ResultFactory.createSerializer(ResultFactory.java:75)
    at com.sun.tools.jxc.api.impl.j2s.JAXBModelImpl.generateEpisodeFile(JAXBModelImpl.java:187)
    at com.sun.tools.jxc.ap.SchemaGenerator.process(SchemaGenerator.java:141)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment.callProcessor(JavacProcessingEnvironment.java:794)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment.discoverAndRunProcs(JavacProcessingEnvironment.java:705)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment.access$1800(JavacProcessingEnvironment.java:91)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.run(JavacProcessingEnvironment.java:1035)
    at com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing(JavacProcessingEnvironment.java:1176)
    at com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler.java:1170)
    at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:856)
    at com.sun.tools.javac.main.Main.compile(Main.java:523)
    at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129)
    at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138)
    at com.sun.tools.jxc.SchemaGenerator$Runner.compile(SchemaGenerator.java:261)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.tools.jxc.SchemaGenerator.run(SchemaGenerator.java:166)
    at org.codehaus.mojo.jaxb2.schemageneration.AbstractXsdGeneratorMojo.performExecution(AbstractXsdGeneratorMojo.java:372)
lennartj commented 8 years ago

By the way - the Schema is generated correctly and in the right place. The 'only problem' seems to be that SchemaGen must emit the episode file in the correct path.

I believe that the extra URLEncoding happens in SchemaGenerator$Runner, where the following code is found:

Iterable<? extends JavaFileObject> compilationUnits = fileManager.getJavaFileObjectsFromFiles(options.getFiles());

... which , in turn, calls the constructor for RegularFileObject with the file as its argument. The code for RegularFileObject performs URL/normalize conversion, and the problem occurs.

public URI toUri() {
        return this.file.toURI().normalize();
    }

    public String getName() {
        return this.file.getPath();
    }
lennartj commented 8 years ago

This seems not to be a problem of the JAXB plugin, but of how SchemaGen's internal implementation treats file names.

We could possibly perform some plugin magic, to write the generated episode file to a temporary directory and then move it to its final destination. However, this is a problem/bug/feature which should be raised with the folks developing SchemaGen.

For now, I will document that SchemaGen has problems dealing with whitespace in paths so we can make a release of the plugin.

lennartj commented 6 years ago

Schemagen / JDK bug.

Lonzak commented 5 years ago

We also experience this problem. Has there been a bug entered at schemagen/JDK? If so can the bug be referenced here?

ismailfarukkaya commented 3 years ago

I am experiencing this in 2021, any update in terms of a solution? I understand this was tagged "wontfix" but at least a link to a work around or maybe a dependency upgrade would be appreciated.

EDIT: Well I have found the workaround in the official guide (This is for version 3.0.0 --the latest as of today)

The current workaround is to avoid whitespace in paths whenever you need the plugin’s schemagen goal to work properly.