Handle missing callbacks as entrypoint

[totoR13]

Dear all, I'm using flowdroid to create a call graph of an android application.

In my testing app (available here), there is a class that extends android.os.AsyncTask (I know that it was deprecated in API level 30). In the resulting call graph, the AsyncTask's methods are unreachable (e.g., doInBackground, onPostExecute), despite the execute method is invoked into the app. I tried to add the android.os.AsyncTask class at the end of the AndroidCallbacks.txt file, but I don't solve this issue. How can I handle these callback methods? In addition, how can I retrieve a list of callbacks methods from the SetupApplication object?

Below I report the snipper of my source code.

public class ApkAnalyzer extends Analyzer {
    private final SetupApplication analyzer;

    public ApkAnalyzer(String apkPath) {
        // Reset Soot
        soot.G.reset();

        // Create config for FlowDroid
        InfoflowAndroidConfiguration config = new InfoflowAndroidConfiguration();
        config.getAnalysisFileConfig().setAndroidPlatformDir(Analyzer.ANDROID_JAR.getPath());
        config.getAnalysisFileConfig().setTargetAPKFile(apkPath);
        config.getAnalysisFileConfig().setSourceSinkFile(Analyzer.SOURCE_SINK.getPath());
        config.setCallgraphAlgorithm(InfoflowConfiguration.CallgraphAlgorithm.CHA);
        config.setCodeEliminationMode(InfoflowConfiguration.CodeEliminationMode.NoCodeElimination);
        config.getCallbackConfig().setEnableCallbacks(true);
        config.getCallbackConfig().setCallbackAnalyzer(InfoflowAndroidConfiguration.CallbackAnalyzer.Default);
        config.setEnableExceptionTracking(true);
        config.setMergeDexFiles(true);
        config.setEnableReflection(true);
        this.analyzer = new SetupApplication(config);
        analyzer.setCallbackFile(Analyzer.ANDROID_CALLBACK.getPath());
        analyzer.setCallbackClasses(loadAndroidCallbacks(Analyzer.ANDROID_CALLBACK.getPath()));
    }

    private static Set<String> loadAndroidCallbacks(String callbackFile) {
        Set<String> output = new HashSet<>();
        BufferedReader bufferedReader;
        try {
            bufferedReader = new BufferedReader(new FileReader(callbackFile));
            String line;
            while ((line = bufferedReader.readLine()) != null) {
                if (!line.isEmpty()) {
                    output.add(line);
                }
            }
            bufferedReader.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
        return output;
    }

    @Override
    public CallGraph run() {
        this.analyzer.runInfoflow(Analyzer.SOURCE_SINK.getPath());
        return Scene.v().getCallGraph();
    }
[...]
}

Thank you in advance. Regards

[flankerhqd]

Edit: This issue isn't reproduced on newest dev branch.

[StevenArzt]

The AsyncTask is not a callback, so adding it to AndroidCallbacks.txt won't help. Still, it requires special handling during callgraph construction, since the call to execute leads to edges to other methods such as onPreExecute. We don't have the simple case in which the invoked method is identical to the callee. Similar challenges arise, e.g., for Thread.start().

In Soot, we have the concept of fake edges to handle such cases. The onPreExecute(), onPostExecute etc. methods are simply missing here. You can help us out by extending virtualedges.xml in Soot. Merge requests are always welcome. Just a quick try (I haven't tested it):

<edge type="EXECUTOR">
    <source invoketype="instance" subsignature="void execute(java.lang.Runnable)" />
        <targets>
            <direct subsignature="void onCancelled()" target="base"/>
            <direct subsignature="void onOnPreExecute()" target="base"/>
        </targets>
</edge>

It would be great if you could extend the file, test it with your app, and open a merge request.

[flankerhqd]

The AsyncTask is not a callback, so adding it to AndroidCallbacks.txt won't help. Still, it requires special handling during callgraph construction, since the call to execute leads to edges to other methods such as onPreExecute. We don't have the simple case in which the invoked method is identical to the callee. Similar challenges arise, e.g., for Thread.start().

In Soot, we have the concept of fake edges to handle such cases. The onPreExecute(), onPostExecute etc. methods are simply missing here. You can help us out by extending virtualedges.xml in Soot. Merge requests are always welcome. Just a quick try (I haven't tested it):

<edge type="EXECUTOR">
    <source invoketype="instance" subsignature="void execute(java.lang.Runnable)" />
        <targets>
            <direct subsignature="void onCancelled()" target="base"/>
            <direct subsignature="void onOnPreExecute()" target="base"/>
        </targets>
</edge>

It would be great if you could extend the file, test it with your app, and open a merge request.

Hi Steven:

Yes I think there's already a rule for AsyncTask in https://github.com/soot-oss/soot/blob/3966f565db6dc2882c3538ffc39e44f4c14b5bcf/src/main/resources/virtualedges.xml#L114 . But I'm not sure why this is not working.

I remembered when I test an older version of Soot, this issue does not exist. However for Soot 3.1.0 and newer, this issue arises.

[StevenArzt]

The rule does not point to onPostExecute etc. We have refactored the concept of fake edges in Soot. The new concept is more precise, but you need to extend the definition file.

[flankerhqd]

The rule does not point to onPostExecute etc. We have refactored the concept of fake edges in Soot. The new concept is more precise, but you need to extend the definition file.

Indeed, I've verified on the current dev branch that doInBackground is connected to the Call Graph.

I think the OP means doInBackground is also not connected. This is not the case on current dev branch. However the 2.8.0 Jar in Github Release seems buggy :( (https://github.com/secure-software-engineering/FlowDroid/releases/tag/v2.8). Maybe the OP is using this one.

[flankerhqd]

The rule does not point to onPostExecute etc. We have refactored the concept of fake edges in Soot. The new concept is more precise, but you need to extend the definition file.

I did some digging on AsyncTask's source, the onPreExecute is called direclty in execute:

    @MainThread
    public final AsyncTask<Params, Progress, Result> execute(Params... params) {
        return executeOnExecutor(sDefaultExecutor, params);
    }

    @MainThread
    public final AsyncTask<Params, Progress, Result> executeOnExecutor(Executor exec,
            Params... params) {
        if (mStatus != Status.PENDING) {
            switch (mStatus) {
                case RUNNING:
                    throw new IllegalStateException("Cannot execute task:"
                            + " the task is already running.");
                case FINISHED:
                    throw new IllegalStateException("Cannot execute task:"
                            + " the task has already been executed "
                            + "(a task can be executed only once)");
            }
        }

        mStatus = Status.RUNNING;

        onPreExecute();

        mWorker.mParams = params;
        exec.execute(mFuture);

        return this;
    }

However it seems this isn't reflected in built CallGraph, is it because if virtual edge is specified in the XML for the execute function(execute -> doInBackground), the actual source of execute will not be analyzed again for the CallGraph?

[StevenArzt]

You normally run FlowDroid against the platform JARs from the Android SDK. If you look into these JARs using a decompiler such as JD GUI, you can see that they do not contain an actual implementation. All methods only throw an exception ("not implemented") in these JARs. You only have the real JARs on the device on Android - or you need to extract them from there for the analysis. In the Android SDK, you only have stubs - these JARs are only meant to compile the APKs, not to run them.

Consequently, we need to manually inject the fake edges to make up for the missing semantic.