Open smowton opened 2 months ago
The unusual factor here is the usage of JarFileObject
with JavacTool
. Our Java extractor needs to convert the FileObject
s passed to JavacTool.getTask
into filesystem paths, and currently only supports SimpleFileObject (i.e., a "real" file on disk).
There are three shortcomings of our extractor to address here:
SomeFileObject.toUri
is usable (where SomeFileObject
is the real type of the passed FileObject
instance), and the scenario where it isn't and we try to fall back on parsing FileObject.toString
. In both cases we only currently support SimpleFileObject
and classes that behave substantially like it, but we warn and continue when the toUri
route finds an unexpected result (e.g. a JarFileObject), whereas we raise the hard exception posted above when using the toString
route. This should be made uniform. This is why the OP's fix of passing --add-opens
seems to work -- it moves us from the less-tolerant toString
code to the more-tolerant toUri
code.toUri
route more often -- since the toString
case is normally quite reliable, we hadn't addressed this, but it is easy to walk the class hierarchy and use FileObject.toUri
, and this helps us address point (3) as well:SimpleFileObject
files, e.g. JarFileObject
, at all. I have a draft PR to export these to a temporary directory and try referring to them from there-- if this seems to work for at least some cases it can't be worse than our current code which always fails in such circumstances (except perhaps the outside chance of writing a lot of text to a temporary directory and wasting disk space).Note that even with this fix, the files referenced by the linked Jenkins code would not end up in the CodeQL database. This is because their translator uses JavacTool
for its parse-and-analyse phase only; it doesn't actually call JavacTask.call
, which is our cue to call the CodeQL java extractor. This is probably expected, since we are trying to trace Java build steps, and the source files referenced as JarFileObject
s (e.g. org/codehaus/groovy/runtime/StringGroovyMethods.java
) are not in fact compiled by JavacTool
.
From a comment posted to https://github.com/github/codeql/issues/7535 by @dwnusbaum:
We are using the
java.compiler
module andJavaCompiler
as part of our (Maven) build process. Here is the full stack trace in our case.com.cloudbees.groovy.cps.tool.Driver
is our build code, and everything works fine on Java 17 when not using the CodeQL autobuild.We can work around the issue by adding
--add-opens=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED
to our build script as in https://github.com/jenkinsci/workflow-cps-plugin/pull/917, but we would prefer not to have to do this.For what it's worth, it seems unusual that
com.semmle.extractor.java.interceptors.JavacToolInterceptor
is trying to change the accessibility ofPathFileObject$JarFileObject.toUri()
. The class in question implementsjavax.tools.JavaFileObject
, which implementsjavax.tools.FileObject
, which has a publictoUri()
method, and those interfaces are part of the public API of thejava.compiler
module.