Submission to Cuckoo Ver. 2.0.6 Fails <Need Help>

[buzzkillg230rc]

Maybe someone can help me identify the issue. When i submit a file from viper to cuckoo i receive "Unable to Submit File" I have set the ip address for cuckoo host and web in the conf file. I'm also trying to review the viper.log, but do not see anything being logged despite me enabling verbose logging. Any help is appreciated.

[stuonela]

Hey buzzkillg did you start up the web interface, api and also cuckoo? I bet you are in the same host or same network.

https://cuckoo.sh/docs/usage/web.html I run the webserver like this: cuckoo web runserver 192.168.254.36:8000 https://cuckoo.sh/docs/usage/api.html and the api like this: cuckoo api --host 192.168.254.36 after that I just start up the vmbox in headless mode and run cuckoo and is good to go.

[k41zen]

I get "Error connecting to Cuckoo". However theHive and Cortex can submit without issues.

I've checked the viper.conf file and entry is correct and obviously the API is listening as Cortex can submit. So this isn't an issue with Cuckoo it seems to be an issue with Viper or my config.

Can someone help?

Running Ubuntu 16.04 and running Viper 2.0-dev.

On the Cuckoo side in the api I see:

[19/Mar/2019 20:19:23] "GET /tasks/list HTTP/1.1" 200 -

[ghost]

I'm having the same issue.

Viper web output: [*] Session opened on 2019-03-22 07:18:45 - viper-web - ERROR - views.py:697 - Error: 'sha256' 2019-03-22 07:18:45 - django.server - INFO - basehttp.py:154 - "GET /project/default/file/01b654c15c38a907d9966a5c1515fa201472ef1e3b831062d283e6cec2763e38/cuckoo/ HTTP/1.1" 200 66

Cuckoo API output: [22/Mar/2019 00:18:45] "GET /tasks/list HTTP/1.1" 200 -

[k41zen]

The issue is in this line of code in views.py (in particular that which is in bold). However, I'm not a python guru. I suspect it's because the API has changed but if anyone could look at this it would be much appreciated.

task_list_filtered = [x for x in task_list["tasks"] if x["sample"]["sha256"] == sha256]

If I comment the checks out which search for the sha256 hash to see if the sample already has been seen and reported in Cuckoo I can submit from Viper to my hearts content. I can't obviously provide the link for the report.

[frennkie]

@k41zen @buzzkillg230rc @icepaule This issue and #703 are the regarding the same problem, right?!

I currently don't have a Cuckoo installation running.. Could one of you check and verify the format of the response from the Cuckoo API?

According to the documentation (https://cuckoo.sh/docs/usage/api.html#tasks-list) this should be:

{
    "tasks": [
        {
            "category": "url",
            "machine": null,
            "errors": [],
            "target": "http://www.malicious.site",
            "package": null,
            "sample_id": null,
            "guest": {},
            "custom": null,
            "owner": "",
            "priority": 1,
            "platform": null,
            "options": null,
            "status": "pending",
            "enforce_timeout": false,
            "timeout": 0,
            "memory": false,
            "tags": []
            "id": 1,
            "added_on": "2012-12-19 14:18:25",
            "completed_on": null
        },
        {
            "category": "file",
            "machine": null,
            "errors": [],
            "target": "/tmp/malware.exe",
            "package": null,
            "sample_id": 1,
            "guest": {},
            "custom": null,
            "owner": "",
            "priority": 1,
            "platform": null,
            "options": null,
            "status": "pending",
            "enforce_timeout": false,
            "timeout": 0,
            "memory": false,
            "tags": [
                        "32bit",
                        "acrobat_6",
                    ],
            "id": 2,
            "added_on": "2012-12-19 14:18:25",
            "completed_on": null
        }
    ]
}

This contains "sample_id": null, which does not make much sense to me.

Does sample_id contains the Viper SHA256 hash in your cases? If so then we only need to change the line into [x for x in task_list["tasks"] if x["sample_id"] == sha256]

[k41zen]

Hey @frennkie

Yes they are duplicate issues. I’ll test this out tonight. Thanks for having a look. Will let you know.

Edit: [sample_id] is null for me too.

K41zen

[k41zen]

Looking at the REST API for /files/view:

GET /files/view/md5/ (str: md5)

GET /files/view/sha256/ (str: sha256)

GET /files/view/id/ (int: id)

Returns details on the file matching either the specified MD5 hash, SHA256 hash or ID.

However, running this using curl, whilst it does return a sample back with the correct sha256 hash seems to have the wrong id.

[frennkie]

I think you are right... We should be using /files/view/<sha256>/. I will look into it tomorrow.

[k41zen]

Great thank you. After building task_list I've simply printed task_list and for a sample I get this:

{'started_on': '2019-03-23 20:07:00', 'errors': [''], 'priority': 1, 'machine': '', 'timeout': 0, 'id': 104, 'options': {}, 'guest': {'status': 'stopped', 'task_id': 104, 'manager': 'VirtualBox', 'name': 'Windows7', 'label': 'Windows7', 'started_on': '2019-03-23 20:07:01', 'id': 99, 'shutdown_on': '2019-03-23 20:10:19'}, 'completed_on': '2019-03-23 20:10:24', 'clock': '2019-03-23 20:03:07', 'tags': [], 'category': 'file', 'target': '/tmp/cuckoo-tmp-cuckoo/tmpxnFjxV/mac.exe', 'custom': '', 'package': '', 'owner': '', 'status': 'reported', 'enforce_timeout': False, 'sample': {'sha256': '06e3beae5bdd477f658f11013df015aa65f971ea5be2238856022eba7d3112dd', 'sha1': '5a3afb57d87c77c07d1b01ea7f306e3e7ed8bc68', 'sha512': '590da9c20a8b5c7b4bc5b26c1b0a0863d83b662279627df557fbe442641afbcba7ef5d4cbd4d05838a8a13a887a1dee8c83c72fbe57f98e7b87853e271732c43', 'file_size': 2071552, 'md5': '7b29c3f38cab047488f259d6cfb29948', 'ssdeep': '24576:HAHnh+eWsN3skA4RV1Hom2KXMmHaZz1XaJKxo/ppDKdiMcCVf/6GR4cR5:6h+ZkldoPK8YaZsC', 'id': 31, 'file_type': 'PE32 executable (GUI) Intel 80386, for MS Windows', 'crc32': '4A493707'}, 'processing': 'p1', 'route': 'internet', 'duration': 203, 'platform': '', 'sample_id': 31, 'memory': False, 'submit_id': None, 'added_on': '2019-03-23 20:03:07'}, {'started_on': '2019-03-23 20:10:25', 'errors': [''], 'priority': 1, 'machine': '', 'timeout': 0, 'id': 105, 'options': {}, 'guest': {'status': 'stopped', 'task_id': 105, 'manager': 'VirtualBox', 'name': 'Windows7', 'label': 'Windows7', 'started_on': '2019-03-23 20:10:25', 'id': 100, 'shutdown_on': '2019-03-23 20:13:48'}, 'completed_on': '2019-03-23 20:13:48', 'clock': '2019-03-23 20:06:12', 'tags': [], 'category': 'file', 'target': '/tmp/cuckoo-tmp-cuckoo/tmpmalEcA/mac.exe', 'custom': '', 'package': '', 'owner': '', 'status': 'reported', 'enforce_timeout': False, 'sample': {'sha256': '06e3beae5bdd477f658f11013df015aa65f971ea5be2238856022eba7d3112dd', 'sha1': '5a3afb57d87c77c07d1b01ea7f306e3e7ed8bc68', 'sha512': '590da9c20a8b5c7b4bc5b26c1b0a0863d83b662279627df557fbe442641afbcba7ef5d4cbd4d05838a8a13a887a1dee8c83c72fbe57f98e7b87853e271732c43', 'file_size': 2071552, 'md5': '7b29c3f38cab047488f259d6cfb29948', 'ssdeep': '24576:HAHnh+eWsN3skA4RV1Hom2KXMmHaZz1XaJKxo/ppDKdiMcCVf/6GR4cR5:6h+ZkldoPK8YaZsC', 'id': 31, 'file_type': 'PE32 executable (GUI) Intel 80386, for MS Windows', 'crc32': '4A493707'}, 'processing': 'p2', 'route': 'internet', 'duration': 202, 'platform': '', 'sample_id': 31, 'memory': False, 'submit_id': None, 'added_on': '2019-03-23 20:06:12'}]

It doesn't have anything to do with the '{' before sha256 does it? When using the curl command for this it returned "31" rather than the actual job task of 104.

[frennkie]

Thanks for posting this.

'sample': **{'**

These double asterisks (**) actually look like Cuckoo bugs to me.. 🤔

[k41zen]

No sorry the double asterisks were me trying to put it in bold. I've put it back now

[frennkie]

Ah... Sorry.

If that is really the result then Cuckoo actually doesn't seem to have changed and our code should work.

Maybe we need to catch some errors on additional content of the response.

[k41zen]

More than happy to help where I can. Happy to give you access to this dev Cuckoo instance too if you need it.

[k41zen]

@frennkie anything I can do my side with this issue? More than happy to help.

[frennkie]

@k41zen sorry for not coming back to you earlier - I had other things I needed to do.

I just re-activated my dev setup including Cuckoo 2.0.6. I can not reproduce your issue. Could it be that is local problem with your cuckoo database? Maybe something went wrong during the upgrade?

If you paste/send me the output of your http[s]://host:port/tasks/list then I could check if / what error Viper needs to catch to handle this more gracefully.

[k41zen]

@frennkie no worries. Grateful for your time in looking at this.

Here's the output from the print for something I tried to submit this morning:

#705 - tasks-list-output.txt

[frennkie]

I retrieved the file. I would assume that replacing the following line with this block should fix it:

In web/viperweb/views.py on line 691:

                task_list_filtered = [x for x in task_list["tasks"] if x["sample"]["sha256"] == sha256]

replace by:

                task_list_filtered = []
                for x in task_list["tasks"]:
                    if hasattr(x["sample"], "sha256"):
                        if x["sample"]["sha256"] == sha256:
                            task_list_filtered.append(x)

Can you check/confirm?

[k41zen]

@frennkie thanks for this. This now allows you to submit samples to Cuckoo. But when I click the Cuckoo button after Cuckoo has finished, it doesn't find that the sample has already been submitted and therefore doesn't provide you with the link to the already finished report.

So I'm guessing the check is still failing.