CuckooPackageError: Unable to execute the initial process, analysis aborted.

[cyjcyjcyj]

My issue is:

cuckoo sandbox always caught an exception : Unable to execute the initial process, analysis aborted.

My Cuckoo version and operating system are:cuckoo2.0.6，ubuntu18.0.4

The log, error, files etc can be found at:

2018-09-28 18:38:07,712 [cuckoo.core.guest] INFO: Starting analysis on guest (id=cuckoo sandbox, ip=192.168.56.101) 2018-09-28 18:38:09,757 [cuckoo.core.guest] INFO: Guest is running Cuckoo Agent 0.8 (id=cuckoo sandbox, ip=192.168.56.101) 2018-09-28 18:38:16,032 [cuckoo.core.guest] WARNING: cuckoo sandbox: analysis caught an exception Traceback (most recent call last): File "C:/tmpxrmyhd/analyzer.py", line 800, in success = analyzer.run() File "C:/tmpxrmyhd/analyzer.py", line 652, in run pids = self.package.start(self.target) File "C:\tmpxrmyhd\modules\packages\exe.py", line 23, in start return self.execute(path, args=shlex.split(args)) File "C:\tmpxrmyhd\lib\common\abstracts.py", line 166, in execute "Unable to execute the initial process, analysis aborted." CuckooPackageError: Unable to execute the initial process, analysis aborted.

2018-09-28 18:38:26,591 [cuckoo.machinery.virtualbox] INFO: Successfully generated memory dump for virtual machine with label cuckoo sandbox to path /home/sun/.cuckoo/storage/analyses/33/memory.dmp [x64] Gathering all referenced SSDTs from KeAddSystemServiceTable... Finding appropriate address space for tables... 2018-09-28 18:40:14,198 [cuckoo.core.scheduler] INFO: Task #33: reports generation completed 2018-09-28 18:40:14,294 [cuckoo.core.scheduler] INFO: Task #33: analysis procedure completed

I have seen the similar issue such as #572, but I still can't resolve this problem. Could anyone help me,please?

[RicoVZ]

Hi cyjcyjcyj,

Thanks for posting an issue.

Does this happen for each file you submit and can you share some of those files with us? :smile:

[cyjcyjcyj]

cyj-files.zip

I have uploading some files I used. For each file, I always got the same errors as I submit above.

[soutzis]

This happens on my setup sporadically, only for a few binaries. I cannot figure out why!:

Issue is also mentioned at #3069 and at #2995

Does anyone have any insights to the cause of this?

[timiuaaa]

Hi Soutzis, Did you find any fix to the above issue?

[soutzis]

Hi Soutzis, Did you find any fix to the above issue?

Hello. Unfortunately, due to the time restrictions I had for the research project I was working on, I didn't have time to properly debug this.

So, I came up a with a simple (yet not so efficient) workaround. I was analysing large batches of malware and I noticed that if the analysis of a binary failed (and therefore the processing of the generated report would fail), there would be either no analysis.log file, or no .json file, or no BSON streams. So I wrote a function that would check the analyses directory and based on the above criteria, it would check if an analysis had succeeded or not (checked before processing!!!!) . If the analysis had succeeded, it would delete unnecessary files, such as dropped files. If the analysis had failed, it would delete all the analysis files, generate the command to resubmit the file and add it to a .txt file. Here is the code snippet if it helps. Feel free to modify it to suit your needs.

def proc_results_checker(analyses_dir: str, post_process: bool = False):
    """
    Set post_process parameter to true, ONLY after a task has finished both analysing and processing.
    Will delete any unnecessary files after an analysis has been processed. The files
    associated with an analysis can cause unnecessary clutter, especially in the case of ransomware 
    where there can be multiple hundreds of BSON streams and thousands of 
    files (due to encrypting everything).
    :param analyses_dir: The directory holding the analyses to inspect 
    :param post_process: If this value is set to True, all useless files and dirs of a successful
                                     report will be deleted
    """
    reanalysis_tasks_path = "/other/reanalysis_tasks.txt"
    anal_dirs_template = analyses_dir+"{0}/{1}/"
    anal_files_template = analyses_dir+"{0}/{1}"
    latest_d = "latest"
    buffer_d = "buffer"
    extracted_d = "extracted"
    files_d = "files"
    logs_d = "logs"
    reports_d = "reports"
    memory_d = "memory"
    shots_d = "shots"
    analysis_f = "analysis.log"
    binary_f = "binary"
    cuckoo_f = "cuckoo.log"
    # not to be deleted
    task_f = "task.json"

    # delete unnecessary symlink "latest"
    remove_if_exists(analyses_dir+latest_d, False)
    t_ids = os.listdir(analyses_dir)
    binaries_list = []

    for t_id in t_ids:
        # if logs dir doesn't exist, it means that this task was already processed and 
        # cleaned.(can also use buffer dir)
        if not os.path.exists(anal_dirs_template.format(t_id, logs_d)):
            continue

        analysis_exists = os.path.exists(anal_files_template.format(t_id, analysis_f))
        # hint: maybe remove the empty logs check for performance issues
        logs_is_empty = len(os.listdir(anal_dirs_template.format(t_id, logs_d))) == 0
        reports_is_empty = len(os.listdir(anal_dirs_template.format(t_id, reports_d))) == 0

        # If analysis has failed for some reason, there will not be any BSON streams, or analysis.log files
        if not analysis_exists or logs_is_empty or (reports_is_empty and post_process):
            data = utils.read_json_file(anal_files_template.format(t_id, task_f))
            # get the binary's real path
            binaries_list.append(data['target'])
            # print to stdout before deleting the analysis, as a precaution
            print("FAILED TASK ID: " + t_id)
            try:
                shutil.rmtree(analyses_dir + t_id)
            except (IOError, OSError) as e:
                print("Error removing directory %s from analyses: %s", t_id, e)
        elif post_process:
            try:
                shutil.rmtree(anal_dirs_template.format(t_id, buffer_d))
                shutil.rmtree(anal_dirs_template.format(t_id, extracted_d))
                shutil.rmtree(anal_dirs_template.format(t_id, files_d))
                shutil.rmtree(anal_dirs_template.format(t_id, logs_d))
                shutil.rmtree(anal_dirs_template.format(t_id, memory_d))
                shutil.rmtree(anal_dirs_template.format(t_id, shots_d))
                os.unlink(anal_files_template.format(t_id, binary_f))
                os.unlink(anal_files_template.format(t_id, cuckoo_f))
            except (IOError, OSError) as e:
                print("Error deleting sub-dirs and files in %s from analyses: %s", t_id, e)

    # write necessary submit commands to file
    if len(binaries_list) > 0:
        with open(reanalysis_tasks_path, utils.FMODE_WRITE) as f:
            for path in binaries_list:
                f.write("cuckoo submit " + path + "\n")
            print("Submit commands for failed tasks written to " + reanalysis_tasks_path)`

[timiuaaa]

Hi Soutzis, Did you find any fix to the above issue?

Hello. Unfortunately, due to the time restrictions I had for the research project I was working on, I didn't have time to properly debug this.

So, I came up a with a simple (yet not so efficient) workaround. I was analysing large batches of malware and I noticed that if the analysis of a binary failed (and therefore the processing of the generated report would fail), there would be either no analysis.log file, or _no .json file_, or no BSON streams. So I wrote a function that would check the analyses directory and based on the above criteria, it would check if an analysis had succeeded or not (checked before processing!!!!) . If the analysis had succeeded, it would delete unnecessary files, such as dropped files. If the analysis had failed, it would delete all the analysis files, generate the command to resubmit the file and add it to a .txt file. Here is the code snippet if it helps. Feel free to modify it to suit your needs.

def proc_results_checker(analyses_dir: str, post_process: bool = False):
    """
    Set post_process parameter to true, ONLY after a task has finished both analysing and processing.
    Will delete any unnecessary files after an analysis has been processed. The files
    associated with an analysis can cause unnecessary clutter, especially in the case of ransomware 
    where there can be multiple hundreds of BSON streams and thousands of 
    files (due to encrypting everything).
    :param analyses_dir: The directory holding the analyses to inspect 
    :param post_process: If this value is set to True, all useless files and dirs of a successful
                                     report will be deleted
    """
    reanalysis_tasks_path = "/other/reanalysis_tasks.txt"
    anal_dirs_template = analyses_dir+"{0}/{1}/"
    anal_files_template = analyses_dir+"{0}/{1}"
    latest_d = "latest"
    buffer_d = "buffer"
    extracted_d = "extracted"
    files_d = "files"
    logs_d = "logs"
    reports_d = "reports"
    memory_d = "memory"
    shots_d = "shots"
    analysis_f = "analysis.log"
    binary_f = "binary"
    cuckoo_f = "cuckoo.log"
    # not to be deleted
    task_f = "task.json"

    # delete unnecessary symlink "latest"
    remove_if_exists(analyses_dir+latest_d, False)
    t_ids = os.listdir(analyses_dir)
    binaries_list = []

    for t_id in t_ids:
        # if logs dir doesn't exist, it means that this task was already processed and 
        # cleaned.(can also use buffer dir)
        if not os.path.exists(anal_dirs_template.format(t_id, logs_d)):
            continue

        analysis_exists = os.path.exists(anal_files_template.format(t_id, analysis_f))
        # hint: maybe remove the empty logs check for performance issues
        logs_is_empty = len(os.listdir(anal_dirs_template.format(t_id, logs_d))) == 0
        reports_is_empty = len(os.listdir(anal_dirs_template.format(t_id, reports_d))) == 0

        # If analysis has failed for some reason, there will not be any BSON streams, or analysis.log files
        if not analysis_exists or logs_is_empty or (reports_is_empty and post_process):
            data = utils.read_json_file(anal_files_template.format(t_id, task_f))
            # get the binary's real path
            binaries_list.append(data['target'])
            # print to stdout before deleting the analysis, as a precaution
            print("FAILED TASK ID: " + t_id)
            try:
                shutil.rmtree(analyses_dir + t_id)
            except (IOError, OSError) as e:
                print("Error removing directory %s from analyses: %s", t_id, e)
        elif post_process:
            try:
                shutil.rmtree(anal_dirs_template.format(t_id, buffer_d))
                shutil.rmtree(anal_dirs_template.format(t_id, extracted_d))
                shutil.rmtree(anal_dirs_template.format(t_id, files_d))
                shutil.rmtree(anal_dirs_template.format(t_id, logs_d))
                shutil.rmtree(anal_dirs_template.format(t_id, memory_d))
                shutil.rmtree(anal_dirs_template.format(t_id, shots_d))
                os.unlink(anal_files_template.format(t_id, binary_f))
                os.unlink(anal_files_template.format(t_id, cuckoo_f))
            except (IOError, OSError) as e:
                print("Error deleting sub-dirs and files in %s from analyses: %s", t_id, e)

    # write necessary submit commands to file
    if len(binaries_list) > 0:
        with open(reanalysis_tasks_path, utils.FMODE_WRITE) as f:
            for path in binaries_list:
                f.write("cuckoo submit " + path + "\n")
            print("Submit commands for failed tasks written to " + reanalysis_tasks_path)`

[timiuaaa]

Appreciate your quick and detailed response, Petros. I have a similar situation where I have to process a large set of benign and malicious samples for research activity. I would appreciate if you can correct my understanding based on your response. The code looks for failed binaries and then put them into a text file (reanalysis_tasks.txt) and then again submit it to cuckoo for processing. Am I correct in my understanding? Thanks

[soutzis]

@timiuaaa the function looks for my own definition of failed analyses, deletes the failed analysis directory completely, and then writes the appropriate "submit" command in a text file. I then manually entered those submit commands for analysis.