search

quarkusio / quarkus

Quarkus: Supersonic Subatomic Java.
https://quarkus.io
Apache License 2.0
13.36k stars 2.56k forks source link

Hot reload fails with InvalidPathException on Windows #3592

Closed dirkweil closed 4 years ago

dirkweil commented 4 years ago

Starting with Quarkus 0.21.0 hot reload failes with InvalidPathException (at least) on Windows.

When a Quarkus application runs in dev mode, i. e. using mvn quarkus:dev, a REST call will trigger a hot reload, if one of the source files has changed before. That worked like a charm up to version 0.20.0. With 0.21.0 the REST response contains an error message:

`

Error restarting Quarkus

<div class="exception-message">
    <h2 class="container">java.nio.file.InvalidPathException: Illegal char &lt;:&gt; at index 2: /C:/Users/dw/.m2/repository/io/quarkus/arc/arc/0.21.0/arc-0.21.0.jar</h2>
</div>

`

As the message complains about the ':' in the file path, I guess, the problem arises on Windows only.

The bug can easily be reproduced:

My environment:

machi1990 commented 4 years ago

To be fixed by https://github.com/quarkusio/quarkus/pull/3561 ? @gsmet

gwenneg commented 4 years ago

Hi @dirkweil, I just tried (from a MacBook) with both 0.21.0 and 0.21.1 (which is the latest Quarkus release) and couldn't reproduce the issue.

machi1990 commented 4 years ago

Hi @dirkweil, I just tried (from a MacBook) with both 0.21.0 and 0.21.1 (which is the latest Quarkus release) and couldn't reproduce the issue.

I guess it is the Windows thingy :-)

gwenneg commented 4 years ago

Oh right, I missed that detail in @dirkweil's message :)

gsmet commented 4 years ago

No, I don't think it's the same issue as #3561. Looks like a different one.

@dirkweil could you provide the full stacktrace? No Windows box here :).

Thanks!

lesha-red commented 4 years ago

Hello!

I'm getting virtually the same walkback, but with the different JAR file:

java.nio.file.InvalidPathException: Illegal char <:> at index 2: /C:/Users/serov/.m2/repository/io/quarkus/quarkus-undertow-common-substitutions/0.21.1/quarkus-undertow-common-substitutions-0.21.1.jar
    at java.base/sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
    at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
    at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
    at java.base/sun.nio.fs.WindowsPath.parse(WindowsPath.java:92)
    at java.base/sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:229)
    at jdk.compiler/com.sun.tools.javac.file.FSInfo.getJarClassPath(FSInfo.java:112)
    at jdk.compiler/com.sun.tools.javac.file.CacheFSInfo.getJarClassPath(CacheFSInfo.java:93)
    at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addJarClassPath(Locations.java:423)
    at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFile(Locations.java:413)
    at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:345)
    at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:352)
    at jdk.compiler/com.sun.tools.javac.file.Locations$SimpleLocationHandler.setPaths(Locations.java:724)
    at jdk.compiler/com.sun.tools.javac.file.Locations.setLocation(Locations.java:2098)
    at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.setLocation(JavacFileManager.java:927)
    at io.quarkus.dev.JavaCompilationProvider.compile(JavaCompilationProvider.java:48)
    at io.quarkus.dev.ClassLoaderCompiler.compile(ClassLoaderCompiler.java:156)
    at io.quarkus.dev.RuntimeUpdatesProcessor.checkForChangedClasses(RuntimeUpdatesProcessor.java:164)
    at io.quarkus.dev.RuntimeUpdatesProcessor.doScan(RuntimeUpdatesProcessor.java:103)
    at io.quarkus.undertow.deployment.devmode.UndertowHotReplacementSetup.handleHotDeploymentRequest(UndertowHotReplacementSetup.java:72)
    at io.quarkus.undertow.deployment.devmode.UndertowHotReplacementSetup$1$1.handleRequest(UndertowHotReplacementSetup.java:61)
    at io.undertow.server.Connectors.executeRootHandler(Connectors.java:376)
    at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
    at io.quarkus.runtime.CleanableExecutor$CleaningRunnable.run(CleanableExecutor.java:224)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
    at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2011)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1535)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1426)
    at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
    at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
    at java.base/java.lang.Thread.run(Thread.java:834)
    at org.jboss.threads.JBossThread.run(JBossThread.java:479)
rikcarve commented 4 years ago

Same here, with 0.21 and 0.21.1 with Windows 7 and 10:

java.nio.file.InvalidPathException: Illegal char <:> at index 2: /C:/Users/rik/.m2/repository/io/quarkus/arc/arc/0.21.0/arc-0.21.0.jar
    at java.base/sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
    at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
    at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
    at java.base/sun.nio.fs.WindowsPath.parse(WindowsPath.java:92)
    at java.base/sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:229)
    at jdk.compiler/com.sun.tools.javac.file.FSInfo.getJarClassPath(FSInfo.java:112)
    at jdk.compiler/com.sun.tools.javac.file.CacheFSInfo.getJarClassPath(CacheFSInfo.java:93)
    at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addJarClassPath(Locations.java:423)
    at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFile(Locations.java:413)
    at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:345)
    at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:352)
    at jdk.compiler/com.sun.tools.javac.file.Locations$SimpleLocationHandler.setPaths(Locations.java:724)
    at jdk.compiler/com.sun.tools.javac.file.Locations.setLocation(Locations.java:2098)
    at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.setLocation(JavacFileManager.java:927)
    at io.quarkus.dev.JavaCompilationProvider.compile(JavaCompilationProvider.java:48)
    at io.quarkus.dev.ClassLoaderCompiler.compile(ClassLoaderCompiler.java:156)
    at io.quarkus.dev.RuntimeUpdatesProcessor.checkForChangedClasses(RuntimeUpdatesProcessor.java:164)
    at io.quarkus.dev.RuntimeUpdatesProcessor.doScan(RuntimeUpdatesProcessor.java:103)
    at io.quarkus.undertow.deployment.devmode.UndertowHotReplacementSetup.handleHotDeploymentRequest(UndertowHotReplacementSetup.java:72)
    at io.quarkus.undertow.deployment.devmode.UndertowHotReplacementSetup$1$1.handleRequest(UndertowHotReplacementSetup.java:61)
    at io.undertow.server.Connectors.executeRootHandler(Connectors.java:376)
    at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
    at io.quarkus.runtime.CleanableExecutor$CleaningRunnable.run(CleanableExecutor.java:224)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
    at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2011)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1535)
    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1426)
    at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
    at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
    at java.base/java.lang.Thread.run(Thread.java:834)
    at org.jboss.threads.JBossThread.run(JBossThread.java:479)
dirkweil commented 4 years ago

There is no error shown in the log; it says only

2019-08-21 10:55:14,638 INFO [io.qua.dev] (executor-thread-1) Changed source files detected, recompiling [C:\GEDOPLAN\projects\gedoplan\showcase\quarkus-demo\quarkus-getting-started\src\main\java\de\gedoplan\showcase\service\GreetingService.java]

The error message is in the response of the REST call triggering hot reload - see attached file. errorResponse.txt

rikcarve commented 4 years ago

I guess it has something to do with commit c2949cb2f4a375b968a0f57737c71001b951724a. Particular with ClassLoaderCompiler line 56ff:

        //this is pretty yuck, but under JDK11 the URLClassLoader trick does not work
        Enumeration<URL> manifests = classLoader.getResources("META-INF/MANIFEST.MF");
        while (manifests.hasMoreElements()) {
            URL url = manifests.nextElement();
            if (url.getProtocol().equals("jar")) {
                String path = url.getPath();
                if (path.startsWith("file:")) {
                    path = path.substring(5, path.lastIndexOf('!'));
                    urls.add(new File(URLDecoder.decode(path, StandardCharsets.UTF_8.name())).toURL());
                }
            }
        }

My local META-INF/MANIFEST.MF e.g. contains this Class-Path: "/C:/Users/rik/.m2/repository/io/quarkus/arc/arc/0.21.0/arc-0.21.0.jar"

dmlloyd commented 4 years ago

I have a preliminary fix at https://github.com/dmlloyd/quarkus/tree/fix-3592 which I am still testing for this.

dmlloyd commented 4 years ago

Looks like something more subtle is going on. Working on it but I'm on a Windows VM and I'm not skilled with Windows, so your continued patience is appreciated :)

dmlloyd commented 4 years ago

I think I figured out what's happening.

The javac tool classes have their own handler for class paths. When you compile against a JAR on the class path, it apparently is recursively searching the manifests of each of those JARs and adding the dependency JARs as well.

However one (or more) of the 100+ JAR files on the class path during this point has a Class-Path manifest attribute which contains a file: URL. But com.sun.tools.javac.file.FSInfo#getJarClassPath doesn't support it: it doesn't parse as a relative URI per spec, rather it just mashes it in as a path to the compiler.

So the next step is to find & fix the JAR that has the bogus information. AFIACT there's no way to "clean" the data before the javac tool gets it.

dmlloyd commented 4 years ago

I also get failing tests on Windows because the generated surefirebooter#######.jar contains Class-Path: file:/C:/Users/... already.

dmlloyd commented 4 years ago

For posterity I'll add another reference here to https://github.com/quarkusio/quarkus/issues/1673#issuecomment-477205320 about proper URL handling for class paths...

dmlloyd commented 4 years ago

See also https://issues.apache.org/jira/browse/SUREFIRE-1588 and https://issues.apache.org/jira/browse/SUREFIRE-1593

dmlloyd commented 4 years ago

There seems to be no solution to the surefire issue, at least before 3.0.0 is released (unless they decide to backport). But assuming a prebuilt Quarkus, we should be able to solve the part where we're writing absolute URIs into our own class paths, wherever that may be taking place.

kodgezgini commented 4 years ago

Problem still exists at Quarkus 0.21.2.

Error restarting Quarkus java.nio.file.InvalidPathException: Illegal char <:> at index 2: /C:/Users/Barbaros/.m2/repository/io/quarkus/arc/arc/0.21.2/arc-0.21.2.jar

java.nio.file.InvalidPathException: Illegal char <:> at index 2: /C:/Users/Barbaros/.m2/repository/io/quarkus/arc/arc/0.21.2/arc-0.21.2.jar at java.base/sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182) at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153) at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77) at java.base/sun.nio.fs.WindowsPath.parse(WindowsPath.java:92) at java.base/sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:229) at jdk.compiler/com.sun.tools.javac.file.FSInfo.getJarClassPath(FSInfo.java:112) at jdk.compiler/com.sun.tools.javac.file.CacheFSInfo.getJarClassPath(CacheFSInfo.java:93) at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addJarClassPath(Locations.java:423) at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFile(Locations.java:413) at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:345) at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:352) at jdk.compiler/com.sun.tools.javac.file.Locations$SimpleLocationHandler.setPaths(Locations.java:724) at jdk.compiler/com.sun.tools.javac.file.Locations.setLocation(Locations.java:2098) at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.setLocation(JavacFileManager.java:927) at io.quarkus.dev.JavaCompilationProvider.compile(JavaCompilationProvider.java:48) at io.quarkus.dev.ClassLoaderCompiler.compile(ClassLoaderCompiler.java:156) at io.quarkus.dev.RuntimeUpdatesProcessor.checkForChangedClasses(RuntimeUpdatesProcessor.java:164) at io.quarkus.dev.RuntimeUpdatesProcessor.doScan(RuntimeUpdatesProcessor.java:103) at io.quarkus.undertow.deployment.devmode.UndertowHotReplacementSetup.handleHotDeploymentRequest(UndertowHotReplacementSetup.java:72) at io.quarkus.undertow.deployment.devmode.UndertowHotReplacementSetup$1$1.handleRequest(UndertowHotReplacementSetup.java:61) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:376) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) at io.quarkus.runtime.CleanableExecutor$CleaningRunnable.run(CleanableExecutor.java:224) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:2011) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1535) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1426) at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29) at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29) at java.base/java.lang.Thread.run(Thread.java:834) at org.jboss.threads.JBossThread.run(JBossThread.java:479)

Tested in;

gsmet commented 4 years ago

@dmlloyd any idea? I checked and your commit is part of 0.21.2.

dmlloyd commented 4 years ago

The remaining issue should be fixed with #3934 - at least, I can't get it to fail locally now...

rikcarve commented 4 years ago

Still get the error in 0.22.0 in Windows 10 Pro and OpenJDK 11. java.nio.file.InvalidPathException: Illegal char <:> at index 2: /C:/Users/rik/.m2/repository/io/quarkus/arc/arc/0.22.0/arc-0.22.0.jar at java.base/sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182) at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153) at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77) at java.base/sun.nio.fs.WindowsPath.parse(WindowsPath.java:92) at java.base/sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:229) at jdk.compiler/com.sun.tools.javac.file.FSInfo.getJarClassPath(FSInfo.java:112) at jdk.compiler/com.sun.tools.javac.file.CacheFSInfo.getJarClassPath(CacheFSInfo.java:93) at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addJarClassPath(Locations.java:423) at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFile(Locations.java:413) at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:345) at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:352) at jdk.compiler/com.sun.tools.javac.file.Locations$SimpleLocationHandler.setPaths(Locations.java:724) at jdk.compiler/com.sun.tools.javac.file.Locations.setLocation(Locations.java:2098) at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.setLocation(JavacFileManager.java:927) at io.quarkus.dev.JavaCompilationProvider.compile(JavaCompilationProvider.java:48)

dmlloyd commented 4 years ago

Commit df51bee0c05d7f9dcdedc315b63d08465e20cbd6 has not yet been included within a release as far as I can tell. The next release should have it.

marcelhanser commented 4 years ago

I get the same error with 0.23.1 in Windows 10 and OpenJDK 11 with a fresh generated quarkus application. I just switched the version to 0.23.1: java.nio.file.InvalidPathException: Illegal char <:> at index 2: /C:/Users/Marcel/.m2/repository/io/quarkus/quarkus-netty/0.23.1/quarkus-netty-0.23.1.jar at java.base/sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182) at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153) at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77) at java.base/sun.nio.fs.WindowsPath.parse(WindowsPath.java:92) at java.base/sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:229) at jdk.compiler/com.sun.tools.javac.file.FSInfo.getJarClassPath(FSInfo.java:112) at jdk.compiler/com.sun.tools.javac.file.CacheFSInfo.getJarClassPath(CacheFSInfo.java:93) at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addJarClassPath(Locations.java:423) at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFile(Locations.java:413) at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:345) at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:352) at jdk.compiler/com.sun.tools.javac.file.Locations$SimpleLocationHandler.setPaths(Locations.java:724) at jdk.compiler/com.sun.tools.javac.file.Locations.setLocation(Locations.java:2098) at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.setLocation(JavacFileManager.java:927) at io.quarkus.dev.JavaCompilationProvider.compile(JavaCompilationProvider.java:48) at io.quarkus.dev.ClassLoaderCompiler.compile(ClassLoaderCompiler.java:157) at io.quarkus.dev.RuntimeUpdatesProcessor.checkForChangedClasses(RuntimeUpdatesProc

geoand commented 4 years ago

Reopening this since we have had a few reports suggesting it's still a problem in 0.23.2

dufoli commented 4 years ago

hello, I have created a pull request for that :

4498

geoand commented 4 years ago

Could the folks that reported this issue perhaps try out #4498 and see if it addresses the issue?

jaikiran commented 4 years ago

@dmlloyd, I was able to get my hands on a Windows system and I was able to reproduce this consistently and narrow this down. An important higher level detail - this is reproducible only on recent (definitely on Java 11 but probably since Java 9?) JDK versions. It probably is because the code in recent JDK versions of com.sun.tools.javac.file.FSInfo#getJarClassPath changed from something like:

for (StringTokenizer st = new StringTokenizer(path); st.hasMoreTokens(); ) {
                String elt = st.nextToken();
                File f = (parent == null ? new File(elt) : new File(parent, elt));
                list.add(f);
            }

to

for (StringTokenizer st = new StringTokenizer(path);
                 st.hasMoreTokens(); ) {
                String elt = st.nextToken();
                Path f = FileSystems.getDefault().getPath(elt);
                if (!f.isAbsolute() && parent != null)
                    f = parent.resolve(f).toAbsolutePath();
                list.add(f);
            }

(usage of Path APIs)

More details follow:

(previously in this issue, you commented):

I think I figured out what's happening.

The javac tool classes have their own handler for class paths. When you compile against a JAR on the class path, it apparently is recursively searching the manifests of each of those JARs and adding the dependency JARs as well.

From what I see in the JDK code, yes, this I believe is what's happening.

(previously in this issue, you commented):

However one (or more) of the 100+ JAR files on the class path during this point has a Class-Path manifest attribute which contains a file: URL. But com.sun.tools.javac.file.FSInfo#getJarClassPath doesn't support it: it doesn't parse as a relative URI per spec, rather it just mashes it in as a path to the compiler.

Partially correct. What seems to be happening is - there's one (and just one I think) of those 100+ jars on the classpath which has a Class-Path manifest attribute of the form:

Class-Path: /C:/Users/Administrator/.m2/repository/io/quarkus/arc/arc/0.23.2/arc-0.23.2.jar /C:/Users/Administrator/.m2/repository/io/quarkus/quarkus-arc/0.23.2/quarkus-arc-0.23.2.jar

(rest of the entries snipped, since the pattern is similar)

As you rightly note - (in recent versions of JDK), the com.sun.tools.javac.file.FSInfo#getJarClassPath uses FileSystems.getDefault().getPath(elt) (where elt is each entry in that Class-Path attribute). This getPath API doesn't support the path parsing if it starts with the / character like above.

The only (I think) jar file which has this kind of Class-Path entries is our xxx-dev.jar file that we generate in the dev mojo in the code here https://github.com/quarkusio/quarkus/blob/master/devtools/maven/src/main/java/io/quarkus/maven/DevMojo.java#L495.

I'm not an expert when it comes to the URI API (it has always confused me and I have had to keep going back to experiemnts to understand how it behaves), but I've read the linked comment here https://github.com/quarkusio/quarkus/issues/1673#issuecomment-477205320. I'm not too sure why the getRawPath is used. From what I have debugged so far on Windows, the getRawPath() actually returns (at least on Windows) a value of the form /C:/Users/Administrator/.m2/repository/io/quarkus/arc/arc/0.23.2/arc-0.23.2.jar (notice the starting /). Because of that, I haven't seen it ever enter this if block here https://github.com/quarkusio/quarkus/blob/master/devtools/maven/src/main/java/io/quarkus/maven/DevMojo.java#L497. So, that Windows check is virtually a no-op. But of course, that isn't causing the problem here - the usage of getRawPath seems to be the root cause. Are we using it so as to let it handle the space characters (by encoding it)?

Keeping all this aside, I think this is a bug in the JDK code itself. Specifically, the public List<Path> getJarClassPath(Path file) throws IOException in com.sun.tools.javac.file.FSInfo is throwing an exception (java.nio.file.InvalidPathException) which isn't declared in its throws clause. I think it should just catch this java.nio.file.InvalidPathException and rethrow it as an IOException so that callers of this getJarClassPath can then catch it and just log it and move on like in the case of Locations.addJarClassPath here https://github.com/openjdk/jdk/blob/master/src/jdk.compiler/share/classes/com/sun/tools/javac/file/Locations.java#L425. This is an internal API of the JDK, so I don't know if this will be accepted as a bug, but I'll raise it in the OpenJDK compiler-dev list and ask for inputs.

dmlloyd commented 4 years ago

Partially correct. What seems to be happening is - there's one (and just one I think) of those 100+ jars on the classpath which has a Class-Path manifest attribute of the form:

Class-Path: /C:/Users/Administrator/.m2/repository/io/quarkus/arc/arc/0.23.2/arc-0.23.2.jar /C:/Users/Administrator/.m2/repository/io/quarkus/quarkus-arc/0.23.2/quarkus-arc-0.23.2.jar

(rest of the entries snipped, since the pattern is similar)

That's expected/normal. This is a relative URI (even though it's an absolute path... a relative URI is one with no "scheme" component), so according to the JAR specification, this is allowed.

As you rightly note - (in recent versions of JDK), the com.sun.tools.javac.file.FSInfo#getJarClassPath uses FileSystems.getDefault().getPath(elt) (where elt is each entry in that Class-Path attribute). This getPath API doesn't support the path parsing if it starts with the / character like above.

The only (I think) jar file which has this kind of Class-Path entries is our xxx-dev.jar file that we generate in the dev mojo in the code here https://github.com/quarkusio/quarkus/blob/master/devtools/maven/src/main/java/io/quarkus/maven/DevMojo.java#L495.

I'm not an expert when it comes to the URI API (it has always confused me and I have had to keep going back to experiemnts to understand how it behaves), but I've read the linked comment here #1673 (comment). I'm not too sure why the getRawPath is used. From what I have debugged so far on Windows, the getRawPath() actually returns (at least on Windows) a value of the form /C:/Users/Administrator/.m2/repository/io/quarkus/arc/arc/0.23.2/arc-0.23.2.jar (notice the starting /). Because of that, I haven't seen it ever enter this if block here https://github.com/quarkusio/quarkus/blob/master/devtools/maven/src/main/java/io/quarkus/maven/DevMojo.java#L497. So, that Windows check is virtually a no-op. But of course, that isn't causing the problem here - the usage of getRawPath seems to be the root cause. Are we using it so as to let it handle the space characters (by encoding it)?

Yes, it handles the URI encoding for us, essentially creating a relative URI as the JAR file specification demands. Note that the block you mentioned is not meant to be entered for the /C: case, it's meant to be entered in the C: case (because C:/foo/bar is syntactically indistinguishable from an absolute URI, which is disallowed; by prepending a / we make it into a relative URI).

Keeping all this aside, I think this is a bug in the JDK code itself. Specifically, the public List<Path> getJarClassPath(Path file) throws IOException in com.sun.tools.javac.file.FSInfo is throwing an exception (java.nio.file.InvalidPathException) which isn't declared in its throws clause. I think it should just catch this java.nio.file.InvalidPathException and rethrow it as an IOException so that callers of this getJarClassPath can then catch it and just log it and move on like in the case of Locations.addJarClassPath here https://github.com/openjdk/jdk/blob/master/src/jdk.compiler/share/classes/com/sun/tools/javac/file/Locations.java#L425. This is an internal API of the JDK, so I don't know if this will be accepted as a bug, but I'll raise it in the OpenJDK compiler-dev list and ask for inputs.

I think they will: the JAR specification does clearly state that the URI is relative. They should be treating the string as a URI rather than a filesystem path; that's the beginning and end of the problem AFAICT.

dmlloyd commented 4 years ago

See https://docs.oracle.com/javase/10/docs/specs/jar/jar.html#main_attributes for the exact spec verbiage. Note that this is the most recent one I could find.

Also: https://docs.oracle.com/javase/10/docs/specs/jar/jar.html#class-path-attribute

dmlloyd commented 4 years ago

The most common cause for this kind of issue in the JDK, historically, is mistaking "relative URL" to mean "relative path" as the authors of the bad javac code seem to have done.

geoand commented 4 years ago

Is there any easy workaround we can apply on our side to remedy the problem temporarily?

dmlloyd commented 4 years ago

Maybe we could drop the drive letter out of the class path on the dev mode JAR, if the drive letter matches the current drive of the user? It won't fix every case but it'll fix some anyway.

geoand commented 4 years ago

Sounds reasonable and I would assume it would address the most common case.

dmlloyd commented 4 years ago

Allegedly System.getenv("SystemDrive") will yield that info.

geoand commented 4 years ago

@jaikiran would you like to give @dmlloyd 's suggestion a shot?

jaikiran commented 4 years ago

@geoand - Yes, I'm experimenting a few ways to get past this, one of them being what @dmlloyd just suggested :) I'll update this thread once I have something.

geoand commented 4 years ago

Great, thanks!

jaikiran commented 4 years ago

@dmlloyd @geoand I decided to go a different approach in fixing this, instead of using the SystemDrive value to decide whether or not to write out the classpathe entries. Given that the real issue is in JDK, plus the fact that using the SystemDrive approach wasn't foolproof (like David already noted), I thought it's better to go with the approach that I propose in https://github.com/quarkusio/quarkus/pull/4527

To summarize the change in that PR - we know that the jar which is causing JDK to run into issues, is the -dev.jar - the one we generate to launch the dev mode JVM. This jar doesn't really have to be added back to the hot deployment compile classpath for reasons I note in the comments of the fix https://github.com/quarkusio/quarkus/pull/4527/files#diff-a9c77c080d4af4da4fb5436598bb7539R104

gsmet commented 4 years ago

So this one should be fixed now, hopefully.

I'm closing it, please reopen if 0.25.0 is (again) not fixing your issue.

Thanks everyone.

rikcarve commented 4 years ago

Just tested 0.25.0 and the issue is still there with JDK11: java.nio.file.InvalidPathException: Illegal char <:> at index 2: /C:/Users/rik/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar It is however fixed for JDK8...

dmlloyd commented 4 years ago

I submitted a JDK bug for the issue where it can't understand drive letters in the URI. I don't have a bug ID yet though.

jaikiran commented 4 years ago

@rikcarve, that's not good :( Can you please paste the stacktrace and a few more details about which Windows OS and what command you use to trigger the build?

rikcarve commented 4 years ago

Sure, here we go. OS: Windows 10 Pro 1903 Maven 3.6.2 Java: AdoptOpenJDK 11.0.4+11

mvn clean compile quarkus:dev

Stacktrace:

java.nio.file.InvalidPathException: Illegal char <:> at index 2: /C:/Users/rik/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar
    at java.base/sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
    at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
    at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
    at java.base/sun.nio.fs.WindowsPath.parse(WindowsPath.java:92)
    at java.base/sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:229)
    at jdk.compiler/com.sun.tools.javac.file.FSInfo.getJarClassPath(FSInfo.java:112)
    at jdk.compiler/com.sun.tools.javac.file.CacheFSInfo.getJarClassPath(CacheFSInfo.java:93)
    at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addJarClassPath(Locations.java:423)
    at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFile(Locations.java:413)
    at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:345)
    at jdk.compiler/com.sun.tools.javac.file.Locations$SearchPath.addFiles(Locations.java:352)
    at jdk.compiler/com.sun.tools.javac.file.Locations$SimpleLocationHandler.setPaths(Locations.java:724)
    at jdk.compiler/com.sun.tools.javac.file.Locations.setLocation(Locations.java:2098)
    at jdk.compiler/com.sun.tools.javac.file.JavacFileManager.setLocation(JavacFileManager.java:927)
    at io.quarkus.dev.JavaCompilationProvider.compile(JavaCompilationProvider.java:48)
    at io.quarkus.dev.ClassLoaderCompiler.compile(ClassLoaderCompiler.java:180)
    at io.quarkus.dev.RuntimeUpdatesProcessor.checkForChangedClasses(RuntimeUpdatesProcessor.java:185)
    at io.quarkus.dev.RuntimeUpdatesProcessor.doScan(RuntimeUpdatesProcessor.java:120)
    at io.quarkus.vertx.http.deployment.devmode.VertxHotReplacementSetup$1.handle(VertxHotReplacementSetup.java:52)
    at io.quarkus.vertx.http.deployment.devmode.VertxHotReplacementSetup$1.handle(VertxHotReplacementSetup.java:44)
    at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$2(ContextImpl.java:316)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:834)

The manifest inside the quarkus-dev.jar looks like this: Manifest-Version: 1.0 Class-Path: /C:/Users/rik/.m2/repository/javax/inject/javax.inject/1/jav ax.inject-1.jar /C:/Users/rik/.m2/repository/javax/annotation/javax.ann

So I guess it is still somehow added to the hotdeploy classpath.

jaikiran commented 4 years ago

The manifest inside the quarkus-dev.jar

This is the jar that's getting generated during the build in the target directory of your project? What is the full directory path where your project is located? Are any symlinks present in this path?

rikcarve commented 4 years ago

c:\Users\rik\dev\github\rikcarve\quarkus>

No symlink, just the standard Users directory

Edit: I even moved the project directory and the maven repo to a newly created path and it still fails.

CSTDev commented 4 years ago

Just to make things more difficult (maybe)... I've tried both JDK11 and 12.0.1 and using 0.25.0 neither is causing this issue for me anymore. This is using the latest getting-started and making some changes, now works fine 😄 (it doesn't if I change back to 0.24.0).

Java Versions:

openjdk version "12.0.1" 2019-04-16
OpenJDK Runtime Environment (build 12.0.1+12)
OpenJDK 64-Bit Server VM (build 12.0.1+12, mixed mode, sharing)

openjdk version "11" 2018-09-25
OpenJDK Runtime Environment 18.9 (build 11+28)
OpenJDK 64-Bit Server VM 18.9 (build 11+28, mixed mode)
rikcarve commented 4 years ago

Weird, just newly cloned the latest getting-started, ran "mvn clean compile quarkus:dev" made a change to the GettingResource.java --> hot deploy failed

jaikiran commented 4 years ago

@rikcarve, How easy or difficult will it be for you to build a custom quarkus locally and then use it to run this app against and get us some logs? I can provide you a repo/branch with some additional debugging log statements which might help us narrow this issue down. I'm glad that the issue was solved for @CSTDev so I think we might have something specific that we need to handle in your case too and would like to understand what that is.

jaikiran commented 4 years ago

@rikcarve Actually, you know what - would it possibel for you to share the -dev.jar that is generated in your target directory somehow? Attach it to this issue or perhaps upload it somewhere and point me to it?

rikcarve commented 4 years ago

Ok, I will try that:

quarkus-dev.zip

rikcarve commented 4 years ago

Hope this helps. I also started mvn with -X to get the debug logs and I could not see the message <Dev mode runner jar " + file + " won't be added to compilation classpath of hot deployment>