Upgrade soot code to 4.3.0 with JDK > 11

[magg]

I use soot on my java project, but I currently need to upgrade my project from Java 8 to 11 or 17, so I would like to know if you can guide me on how to do it, I seem to be hitting lots of errors.

Here's the old code working with java 8 and soot 3.2.1, it was using java 7 in soot.

  G.reset();
            Scene.v().releaseSideEffectAnalysis();

            //define classpath with ConfigClass and java rt.jar
            final String classpath = SootGeneratorServiceImpl.class
                    .getProtectionDomain()
                    .getCodeSource()
                    .getLocation()
                    .getPath()
                    + File.pathSeparator
                    + Paths.get(System.getProperty("java.home")).resolve("lib").resolve("rt.jar");
            Options.v().set_soot_classpath(classpath);
            Options.v().set_java_version(Options.java_version_1_7);
            //define resolve level for some classes we are interested in
            Scene.v().addBasicClass(ILLEGAL_ARGUMENT_EXCEPTION, SootClass.SIGNATURES);
            Scene.v().addBasicClass(CONFIG_CLASS, SootClass.BODIES);
            Scene.v().addBasicClass(CLASS, SootClass.BODIES);
            Scene.v().addBasicClass(EXCEPTION, SootClass.SIGNATURES);
            Scene.v().addBasicClass(STRING, SootClass.SIGNATURES);
            Scene.v().addBasicClass(CONSTRUCTOR, SootClass.BODIES);
            Scene.v().loadNecessaryClasses();

            configClassTemplate = Scene.v().loadClassAndSupport(CONFIG_CLASS);

            //retrieve active bodies for ConfigClass
            for (SootMethod method : configClassTemplate.getMethods()) {
                if (!method.isConcrete()) {
                    continue;
                }
                if (!method.hasActiveBody()) {
                    method.retrieveActiveBody();
                }
            }

What can I do about rt.jar should I download it manually? for JDK 11 and 17?

Apparently there is this line for java 11 but not for java 17

Options.v().set_java_version(Options.java_version_11);

If i remove the resolve rt.jar call

I get

            couldn't find class: java.lang.Object (is your soot-class-path set properly?) Try adding rt.jar to Soot's classpath, e.g.:
            java -cp sootclasses.jar soot.Main -cp .:/path/to/jdk/jre/lib/rt.jar <other options>

Caused by: java.lang.AssertionError
    at soot.SootResolver.resolveClass(SootResolver.java:157)
    at soot.Scene.tryLoadClass(Scene.java:980)
    at soot.Scene.loadBasicClasses(Scene.java:1713)
    at soot.Scene.loadNecessaryClasses(Scene.java:1804)

I am currently running this java jdk and I want to run soot in Java 11. What can I do?

java -version          
openjdk version "11.0.15" 2022-04-19
OpenJDK Runtime Environment (build 11.0.15+10-Ubuntu-0ubuntu0.18.04.1)
OpenJDK 64-Bit Server VM (build 11.0.15+10-Ubuntu-0ubuntu0.18.04.1, mixed mode, sharing)

Is there any plans to support JDK 17 and above?

[StevenArzt]

There is no rt.jar in Java versions newer than 8. Soot has a feature to load classes from the new module file system. Have a look at the ModuleScene.

[magg]

so I changed the code like this it works now

 G.reset();
            Scene.v().releaseSideEffectAnalysis();

            //define classpath with ConfigClass and java rt.jar
            final String classpath = SootGeneratorServiceImpl.class
                    .getProtectionDomain()
                    .getCodeSource()
                    .getLocation()
                    .getPath();
            final String fclasspath = classpath
                    + File.pathSeparator
                    + Paths.get(System.getProperty("java.home")).resolve("lib").resolve("jrt-fs.jar");
            //Options.v().set_soot_classpath(classpath);
            //Options.v().set_java_version(Options.java_version_1_7);

            Options.v().set_soot_classpath("VIRTUAL_FS_FOR_JDK" + File.pathSeparator + classpath);
            //Options.v().set_src_prec(Options.src_prec_class);
            Options.v().set_allow_phantom_refs(true);
            Options.v().set_exclude(excludeList);

            //Options.v().set_prepend_classpath(true);
            //Options.v().set_soot_modulepath(classpath);
            Options.v().set_process_dir(Arrays.asList(fclasspath.split(File.pathSeparator)));
            Options.v().set_java_version(Options.java_version_11);

            //define resolve level for some classes we are interested in
            Scene.v().addBasicClass(ILLEGAL_ARGUMENT_EXCEPTION, SootClass.SIGNATURES);
            Scene.v().addBasicClass(CONFIG_CLASS, SootClass.BODIES);
            Scene.v().addBasicClass(CLASS, SootClass.BODIES);
            Scene.v().addBasicClass(EXCEPTION, SootClass.SIGNATURES);
            Scene.v().addBasicClass(STRING, SootClass.SIGNATURES);
            Scene.v().addBasicClass(CONSTRUCTOR, SootClass.BODIES);
            Scene.v().loadNecessaryClasses();

            configClassTemplate = Scene.v().loadClassAndSupport(CONFIG_CLASS);

 //retrieve active bodies for ConfigClass
            for (SootMethod method : configClassTemplate.getMethods()) {
                if (!method.isConcrete()) {
                    continue;
                }
                if (!method.hasActiveBody()) {
                    method.retrieveActiveBody();
                }
            }

It works but I needed to add Options.v().set_allow_phantom_refs(true); since I'm running it from Maven it loads lots of dependencies and soot can't find dependencies like org.springframework. I'm using soot inside a Spring boot app

Is there a way to include deps?

[StevenArzt]

You should be able to put the library JARs on the classpath along with the virtual fs, as you would with multiple JARs.

[magg]

I also have this class to call soot from the java code using reflection

private void toDexWithClassLoader(
            ClassLoader loader, String jarUri, String dexUri) throws ReflectiveOperationException {

        final Class<?> optionsClass = Class.forName(Options.class.getName(), true, loader);
        final Object options = optionsClass.getMethod("v").invoke(optionsClass);

        optionsClass.getMethod("set_soot_classpath", String.class)
                .invoke(options, "VIRTUAL_FS_FOR_JDK" + File.pathSeparator + jarUri);
        optionsClass.getMethod("set_process_dir", List.class).invoke(options, Collections.singletonList(jarUri));
        optionsClass.getMethod("set_output_dir", String.class).invoke(options, dexUri);
        optionsClass.getMethod("set_output_format", int.class).invoke(options, Options.output_format_dex);
        optionsClass.getMethod("set_java_version", int.class).invoke(options, Options.java_version_1_11);
        optionsClass.getMethod("set_allow_phantom_refs", boolean.class).invoke(options, true);
        optionsClass.getMethod("set_output_jar", boolean.class).invoke(options, true);

        final Class<?> sceneClass = Class.forName(Scene.class.getName(), true, loader);
        final Object scene = sceneClass.getMethod("v").invoke(sceneClass);
        sceneClass.getMethod("loadNecessaryClasses").invoke(scene);

final Class<?> packManagerClass = Class.forName(PackManager.class.getName(), true, loader);
        final Object packManager = packManagerClass.getMethod("v").invoke(packManagerClass);
        packManagerClass.getMethod("runPacks").invoke(packManager);
        packManagerClass.getMethod("writeOutput").invoke(packManager);
}

The writeOutput method fails, I believe an exception is thrown here, my code produces originalApk == null, any idea here @StevenArzt

DexPrinter.java

public void print() {
    try {
      if (Options.v().output_jar()
          || (originalApk != null && Options.v().output_format() != Options.output_format_force_dex)) {
        printZip();
      } else {
        final String outputDir = SourceLocator.v().getOutputDir();
        LOGGER.info("Writing dex files to \"{}\" folder.", outputDir);
        dexBuilder.writeTo(outputDir);
      }
    } catch (IOException e) {
      throw new CompilationDeathException("I/O exception while printing dex", e);
    }

Caused by: soot.CompilationDeathException: I/O exception while printing dex
    at soot.toDex.DexPrinter.print(DexPrinter.java:1715)
    at soot.PackManager.writeDexOutput(PackManager.java:593)
    at soot.PackManager.writeOutput(PackManager.java:573)
    ... 43 more
Caused by: java.io.IOException: Stream closed
    at java.base/java.util.zip.ZipOutputStream.ensureOpen(ZipOutputStream.java:97)
    at java.base/java.util.zip.ZipOutputStream.closeEntry(ZipOutputStream.java:249)
    at soot.toDex.DexPrinter.addManifest(DexPrinter.java:406)
    at soot.toDex.DexPrinter.printZip(DexPrinter.java:315)
    at soot.toDex.DexPrinter.print(DexPrinter.java:1708)
    ... 45 more

[Ilya-Skiba]

This issue is cause by wrapping manifest in try-with-resources block in https://github.com/soot-oss/soot/commit/7066d685a459bef88ca4f630e35cf15ab858c29b. PR created: https://github.com/soot-oss/soot/pull/2077

[StevenArzt]

You don't need to use reflection to use Soot with newer Java versions. @MarcMil has done this before and might have an idea what is going wrong on your end.