connglli / ELEGANT

ELEGANT - a tool to Effectively LocatE fraGmentAtion-iNduced compaTibility issues.
MIT License
3 stars 0 forks source link

weekly report: 2018.02.xx-2018.03.13 #14

Closed connglli closed 6 years ago

connglli commented 6 years ago

周报

这次的周报总结一下这段(挺长)时间做的事情,好久没写了,这次也写的稍微多了一点。

回顾

首先还是说一下上一次周报提到的问题,上次周报提到了这么几个问题:

  1. 对于 reflection-fixing,除了之前提到的利用 if (null == someMethodHandle) 检查的解决方案,还能预想到一种类似的解决方案,利用 try-catch 来空指针等的捕获异常。
  2. 对于 reflection-fixing,之前提交的代码中已经有了对空指针检查的部分,但检查相对较弱,代码中所做的检查仅仅是对每次 someMethodHandle 调用前的是否有空指针检测进行检测,而没有考虑控制流和数据流的问题。正确的处理方式应该是对 someMethodHandle 所在的方法进行 nullness analysis,从而保证在 someMethodHandle 的每次调用前都能够保证 someMethodHandle 的非空性。
  3. 剪枝阶段对调用链的检测没有考虑到安卓生命周期的问题,猜想造成这种问题的原因其实是 event-driven 的方式造成了 CFG 的不完整性,从而使得我们无法获取到正确的 slicing。

解决方案

  1. 上述 1/2 问题可以合并解决,首先对 someMethodHandle 所在的方法添加一个 NullnessAnalysis,如果每次对 someMethodHandle 的调用都能保证其非空性,我们认为开发者已经将这个 issue 进行了 fix;否则,我们将 (1) 检查此处调用是否在一个 try 语句块中,并 (2) 检查该语句块是否是用来捕获 NullPointerException/NoSuchMethodException 的语句块,如果 (1) (2) 同时满足我们同样认为开发者已经对这个 issue 进行了 fix。否则,我们认为开发者未能预见这种情况,我们将汇报该条 issue。下面的代码展示了主逻辑部分(代码一直在我本地,昨天刚刚提交到 github 上,代码主要位于 RFinder.java 里):

    // non-nullness can be ensured by: 1. if-else checking 2. try-catch block
    Body body = caller.getActiveBody();
    
    // here we will run a nullness analysis, to guarantee that the r9 is checked non-nullness via if-else
    NullnessAnalysis nullnessAnalysis = new NullnessAnalysis(new BriefUnitGraph(body));
    
    // we must guarantee that the handler got by reflection is not null
    boolean    fixed = true;
    List<Unit> units = new ArrayList<>(body.getUnits());
    int        index = units.indexOf(callSiteUnit);
    
    // get all try-catch blocks to get ready for try-catch checking
    List<Trap> traps = new ArrayList<>(body.getTraps());
    
    // traverse units after call site, guarantee that invoking of r9 is checked non-nullness
    for (int i = index + 1; i < units.size(); i++) {
       Unit u = units.get(i);
    
       // u does not invoke r9, skip it
       if (!Strings.contains(u.toString(), definedVar.toString() + ".")) { continue; }
    
       // ensure that r9 is non-null via if-else
       if (!nullnessAnalysis.isAlwaysNonNullBefore(u, (Immediate) definedVar)) {
           // non-nullness can not be guaranteed by if-else,
           // then, we must do a try-catch checking
           boolean caught = false;
           for (Trap trap : traps) {
               SootClass exceptionClass = trap.getException();
               String exceptionJavaStyleName = exceptionClass.getJavaStyleName();
               int bidx = units.indexOf(trap.getBeginUnit());
               int eidx = units.indexOf(trap.getEndUnit());
    
               // this trap can catch u, and the exception to be caught
               // is a NullPointerException or a NoSuchMethodException
               // or a its super classes' instance
               if (bidx <= i && i <= eidx &&
                       ("NullPointerException".equals(exceptionJavaStyleName) ||
                        "NoSuchMethodException".equals(exceptionJavaStyleName) ||
                        "ReflectiveOperationException".equals(exceptionJavaStyleName) ||
                        "Exception".equals(exceptionJavaStyleName))
                       ) {
                   caught = true;
                   break;
               }
           }
    
           if (!caught) {
               fixed = false;
               break;
           }
       }
    }
    
    if (!fixed) {
       validatedEdges.add(edge);
    }
  2. 虽然 soot/flowdroid 对安卓有了一个假入口函数 FakeMainEntry 的实现,并使我们能够获取到整个的 CFG,但从我们并不能获得正确的 slicing 这点来看,我们猜想这个 CFG 是不完整的,但这一点目前并没有获得证实(我向 soot-email-list 发了两封邮件询问几个有关 soot 的问题,一封是年前发的,一封是前几天发的,但都没有得到回应,真惨,明明我经常收到作者对其他人询问问题的回复),因此这个问题到现在仍然悬而未决,我现有的想法,或者是请教一下刘烨庞师兄像 GreenDroid (Y. Liu, etc) 一样去对 Android 进行一点建模,或者是像 EnergyPatch (A. Banerjee, etc) 一样利用 Dynodroid 获取 EFG,从而获取 CFG,然后抛弃 soot 获取 slicing 的方法,重新写一个 program slicing 的算法(我目前手边和网上也都没有找到一份完整的算法,我目前看到的 program slicing 的说法都很模糊,没看到十分精确、完整的定义),但这样都会极大地(真的是不止一点点地)增大工作量。==不知道 Lili 对这点有没有什么想法?:confused:==

其他工作

除了对上述提到的问题进行思考和部分着手解决,这段时间还在写脚本整理最后 Lili 进行实验的 27 个 App,打算利用现在的版本跑个测试试试,主要是:

  1. 从 github 上爬。国内 git clone 是真的慢,这 27 个项目单单下载就花接近了 2 天的时间,大项目 git 在下载过程中经常因为网络问题挂,开了 ss 也不稳定,所以脚本里会有 if [-d xxx] 这样检查项目的代码存在,因为这脚本跑了不知道几遍。:cry:
  2. 建分支(Lili 的实验都是基于某个分支的代码,应该是做实验时比较稳定的版本),为方便,这里我统一叫它们 fic-finder-test-version 分支,过程中发现,有两个项目的两个分支不存在(==不知道是不是 Lili 论文里标错了 commit 号==),分别是 "openvpn/9278fa4""owncloud/cfd3b94"
  3. 进行 apk 的构建,这工作量相对来说还可以,这也真是一项挑战,不用 docker 做项目开发的开发者们不知道在配环境上费了多大功夫,期间遇到了不少麻烦,虽然有解决的,但不知道这种粗略的解决会对工程造成多大的影响。

下面先罗列下脚本(不打算上传,所以在这里写一下),然后记录一下期间的问题和解决方案(如果以后真的有问题也方便查看)。

下载 - download.sh

#! /bin/bash

TEST_PROJECTS_REPO_URL=(
  "https://github.com/Integreight/1Sheeld-Android-App.git"
  "https://github.com/connectbot/connectbot.git"
  "https://github.com/ankidroid/Anki-Android.git"
  "https://github.com/AntennaPod/AntennaPod.git"
  "https://github.com/siacs/Conversations.git"
  "https://github.com/cgeo/cgeo.git"
  "https://github.com/xbmc/Kore.git"
  "https://github.com/AnySoftKeyboard/AnySoftKeyboard.git"
  "https://github.com/erickok/transdroid.git"
  "https://github.com/k9mail/k-9.git"
  "https://github.com/r3gis3r/CSipSimple.git"
  "https://github.com/DrKLO/Telegram.git"
  "https://github.com/wordpress-mobile/WordPress-Android.git"
  "https://github.com/OpenVPN/openvpn.git"
  "https://github.com/brave/link-bubble.git"
  "https://github.com/firetech/PactrackDroid.git"
  "https://github.com/moezbhatti/qksms.git"
  "https://github.com/rcgroot/open-gpstracker.git"
  "https://github.com/liato/android-bankdroid.git"
  "https://github.com/evercam/evercam-android.git"
  "https://github.com/bitcoin-wallet/bitcoin-wallet.git"
  "https://github.com/guardianproject/ChatSecureAndroid.git"
  "https://github.com/inaturalist/iNaturalistAndroid.git"
  "https://github.com/hypery2k/owncloud.git"
  "https://github.com/pockethub/PocketHub.git"
  "https://code.videolan.org/videolan/vlc-android.git"
)
readonly TEST_PROJECTS_REPO_URL

# $1: project name to be downloaded
# $2: project repo_url
function download() {
  # some projects have been already downloaded, we ignore them
  if [[ ! -d ${1} ]]; then
    echo -e "\033[1;32mdownload ${1} from ${2}\033[0m"
    # downloads from repo of $2
    git clone ${2}
  fi
}

for repo_url in ${TEST_PROJECTS_REPO_URL[@]}; do
  project_name=$(echo ${repo_url} | cut -d '/' -f 5 | cut -d '.' -f 1)
  download ${project_name} ${repo_url}
done

建立分支 - checkout.sh

#! /bin/bash

FIC_FINDER_TEST_BRANCH_NAME="fic-finder-test-version"

TEST_PROJECTS_COMMIT=(
  "IrssiNotifier/ad68bc3"
  "1Sheeld-Android-App/b49c98a"
  "connectbot/49712a1"
  "Anki-Android/dd654b6"
  "AntennaPod/6f15660"
  "Conversations/1a073ca"
  "vlc-android/4588812"
  "cgeo/5f58482"
  "Kore/0b73228"
  "AnySoftKeyboard/d0be248"
  "transdroid/28786b0"
  "k-9/74c6e76"
  "CSipSimple/fd1e332"
  "Telegram/a7513b3"
  "WordPress-Android/efda4c9"
  # "openvpn/9278fa4"
  "link-bubble/65cd91d"
  "PactrackDroid/1090758"
  "qksms/73b3ec4"
  "open-gpstracker/763d1e2"
  "android-bankdroid/f491574"
  "evercam-android/c4476de"
  "bitcoin-wallet/3235281"
  "ChatSecureAndroid/30f54a4"
  "iNaturalistAndroid/1e837ca"
  # "owncloud/cfd3b94"
  "PocketHub/a6e9583"
)
readonly TEST_PROJECTS_COMMIT

# $1: the directory
# $2: the commit that will be checkouted
function makeBranchOnCommit() {
  # enter the directory
  # make a new branch named "fic-finder-test-version" based on the corresponding commit
  # exit this directory
  echo -e "entering \033[1;32m${1}\033[0m"
  cd ${1}

  git checkout -b ${FIC_FINDER_TEST_BRANCH_NAME} ${2} &&
    echo -e "\033[1;32mmaking branch ${FIC_FINDER_TEST_BRANCH_NAME} based on commit ${2}\033[0m"

  echo -e "exiting \033[1;32m${1}\033[0m"
  cd ..
}

for ele in ${TEST_PROJECTS_COMMIT[@]}; do
  project_name=$(echo ${ele} | cut -d '/' -f 1)
  commit=$(echo ${ele} | cut -d '/' -f 2)
  makeBranchOnCommit ${project_name} ${commit}
done

构建 apk

#! /bin/bash

TESTS_HOME=`pwd`
readonly TESTS_HOME

TEST_PROJECTS_LIST=(
  "IrssiNotifier/Android"
  "1Sheeld-Android-App"
  "connectbot"
  "Anki-Android"
  "AntennaPod"
  "Conversations"
  "vlc-android"
  "cgeo"
  "Kore"
  "AnySoftKeyboard"
  "transdroid"
  "k-9"
  "CSipSimple"
  "Telegram"
  "WordPress-Android"
  # "openvpn"
  "link-bubble/Application"
  "PactrackDroid"
  "qksms"
  "open-gpstracker/studio"
  "android-bankdroid"
  "evercam-android"
  "bitcoin-wallet/wallet"
  "ChatSecureAndroid"
  "iNaturalistAndroid"
  # "owncloud"
  "PocketHub"
)
readonly TEST_PROJECTS_LIST

for project_path in ${TEST_PROJECTS_LIST[@]}; do
  project_name=$(echo ${project_path} | cut -d '/' -f 1)

  echo -e "\033[1;32m========> ${project_name}\033[0m"

  echo "  1. entering"
  cd ${project_path}

  if [[ ! -f "gradlew" ]]; then
    echo "${project_name}" >> "${TESTS_HOME}/not.studio.log"
    echo "  2. exiting"
    cd ${TESTS_HOME}
  else
    # here we use assembleDebug to build a debug version
    chmod +x gradlew &&
      echo "  2. cleaning up" &&
      ./gradlew clean > /dev/null 2> "${TESTS_HOME}/${project_name}.clean.fail.log" &&
      echo "  3. building" &&
      ./gradlew assembleDebug > /dev/null 2>> "${TESTS_HOME}/${project_name}.build.fail.log" &&
      echo -e "  \033[1;32msucceeded\033[0m in building" &&
      echo "${project_name}" >> ${TESTS_HOME}/succ.log

    echo "  5. exiting"
    cd ${TESTS_HOME}
  fi
done

构建过程中遇到的问题

到目前还未能编译成功的问题