search

cuckoosandbox / cuckoo

Cuckoo Sandbox is an automated dynamic malware analysis system
http://www.cuckoosandbox.org
Other
5.53k stars 1.7k forks source link

Ridiculous scores (exceeding 10, or much lower than they should be). #2287

Open ghost opened 6 years ago

ghost commented 6 years ago
My issue is:

Scores aren't working as they should be. (I know this is an alpha feature, however other Cuckoo setups I've used have worked fine, so I don't see why this shouldn't?).

I've run several samples that should have been a 10, and they either exceeded that, or they were super low. They also change every time they're analysed (for example, I threw the same Wannacry sample at it 3 times, and each score was different (20.4, 21.6, 22.4). I also ran a bulk lot of about 700 samples, all of which had scores between 0.5 & 2, when they should all be much higher.

I ran the same same Wannacry sample on someone else's server, and it seems to be fine over there (http://voodooshield.ddns.net:8080/analysis/13358/), though they are running an older version of Cuckoo (or possibly a modified one? I'm not sure).

My Cuckoo version and operating system are:

Cuckoo 2.0.5.3 (installed via pip), Ubuntu Server 18.04 LTS

The log, error, files etc can be found at:

Had a look through the logs (cuckoo.log & each processing log), and all seems good. No errors or anything, other than one about m2crypto, which I'm also trying to fix. Just having issues installing that specific version.

You do not have the m2crypto library installed preventing certificate extraction. Please read the Cuckoo documentation on installing m2crypto (you need SWIG installed and then `pip install m2crypto==0.24.0`)!
RicoVZ commented 6 years ago

Hi aidenatt,

Thanks for posting an issue.

The "out of 10" is kind of useless for now. But the scoring system is working as intented. We do have plans to revamp it. Scores rarely reach 10 now. This is because of the way it is implemented.

The score is calculated by adding up the severity levels of each matched Cuckoo signature and dividing it by 5.0. The difference in score means different signatures (fewer, more or others) are matched. This also means there is no limit to how high the score can be. It all depends on how many signatures are matched.

The signatures are matched against all collected data (behavioral etc). Since behavioral data can differ per analysis, it means different signatures can match. This can result in a different score for analyses of the same file.

This means that 0.5 and 2 scores can very well be because of a few (but still high severity) matched signatures. There certainly is room for improvement. Plans for change exist. :smile:

About the Cuckoo setup you linked: I believe this is an instance of a fork of Cuckoo, called cuckoo-modified.

jbremer commented 6 years ago

Also, provided this is an open source project, feel free to help us improving it.. ;-)

twesterhever commented 6 years ago

@RicoVZ: Scores above 10 occur frequently here. :-)

RicoVZ commented 6 years ago

@twesterhever

What I meant as a reply was that they do not reach 10 when they should. :smile: This is a thing about the scoring we would like to change. If some signatures were triggered that almost certainly mean the file is malicious, it should just be 10. :smile: