search

malicialab / avclass

AVClass malware labeling tool
MIT License
451 stars 115 forks source link

VirusTotal v3 output incompatibility #36

Closed littlepwnie closed 2 years ago

littlepwnie commented 3 years ago

Greetings.

While submitting a file containing multiple json's I received the following error:

$ python avclass2_labeler.py -vt data.jsonl -p -vt3 > out.txt [-] Using tagging rules in /home/user/avclass/avclass2/data/default.tagging [-] Using taxonomy in /home/user/avclass/avclass2/data/default.taxonomy [-] Using expansion tags in /home/user/avclass/avclass2/data/default.expansion [-] Processing input file chris.jsonl [-] 0 JSON readTraceback (most recent call last): File "avclass2_labeler.py", line 489, in <module> main(args) File "avclass2_labeler.py", line 149, in main sample_info = get_sample_info(vt_rep) File "/home/user/avclass/avclass2/lib/avclass2_common.py", line 454, in get_sample_info_vt_v3 scans = vt_rep['attributes']['last_analysis_results'] TypeError: list indices must be integers, not str

I used both Python 3.8.5 and 2.7.18, just in case. I suppose it has to do with the json data. They are all formatted like this and I can't really spot a difference from your sample v3 json:

{"data": [{"attributes": {"type_description": "Win32 EXE", "bytehero_info": "Trojan.Win32.Heur.Gen", "tlsh": "T16753027373B4FFB6DF71E478B4660897BE430510A6905B9B425C6B0AB4E09E42770D32", "vhash": "06402d0d7bz2!z", "trid": [{"file_type": "Win32 Executable (generic)", "probability": 52.9}, {"file_type": "Generic Win/DOS Executable", "probability": 23.5}, {"file_type": "DOS Executable Generic", "probability": 23.5}], "crowdsourced_yara_results": [{"rule_name": "win_alina_pos_auto", "description": "autogenerated rule brought to you by yara-signator", "author": "Felix Bilstein - yara-signator at cocacoding dot com", "ruleset_id": "0085158f09", "ruleset_name": "win.alina_pos_auto", "match_in_subfile": true, "source": "https://malpedia.caad.fkie.fraunhofer.de/"}], "creation_date": 558322502, "names": ["upx_W32_Alina_3_4_B_fsg.exe", "upx_W32_Alina_3_4_B_unpacked.exe"], "last_modification_date": 1619787163, "type_tag": "peexe", "times_submitted": 2, "total_votes": {"harmless": 0, "malicious": 0}, "size": 64409, "popular_threat_classification": {"suggested_threat_label": "trojan.mint/zard", "popular_threat_category": [["trojan", 19]], "popular_threat_name": [["mint", 5], ["zard", 5], ["poscardstealer", 3]]}, "authentihash": "75ee3fb1cb21a10ece4f9b5a9ce12a71c652eb41016d72394dfac58f5dd61373", "last_submission_date": 1619779790, "meaningful_name": "upx_W32_Alina_3_4_B_fsg.exe", "sandbox_verdicts": {"C2AE": {"category": "malicious", "confidence": 70, "sandbox_name": "C2AE", "malware_classification": ["STEALER"], "malware_names": ["AlinaStealer"]}}, "sha256": "bc3df8753583d38f479b6151a106b9acaecf83f209ca01774acbcd4132da1572", "type_extension": "exe", "tags": ["peexe", "fsg"], "last_analysis_date": 1619779790, "unique_sources": 2, "first_submission_date": 1605398863, "sha1": "932db98d066a47a02b4dba14e70fb2eca9d19195", "ssdeep": "1536:nIBrRlBvL2xS+j6gJiLi+zYVldHfucVpMMJao:nIBBvL2xSuNsLNcVldHTVDIo", "packers": {"PEiD": "FSG v2.0 -> bart/xt", "Cyren": "FSG"}, "md5": "22065d825f6445127e6e5aeaa2253521", "pe_info": {"timestamp": 558322502, "entry_point": 340, "machine_type": 332, "imphash": "87bed5a7cba00c7e1f4015f1bdae2183", "sections": [{"name": "", "chi2": -1.0, "virtual_address": 4096, "flags": "rw", "raw_size": 0, "entropy": 0.0, "virtual_size": 409600, "md5": "d41d8cd98f00b204e9800998ecf8427e"}, {"name": "", "chi2": 9074.31, "virtual_address": 413696, "flags": "rw", "raw_size": 63897, "entropy": 7.91, "virtual_size": 65536, "md5": "5b0f4e4301311891db2e4ed19dab1204"}], "import_list": [{"library_name": "KERNEL32.dll", "imported_functions": ["LoadLibraryA", "GetProcAddress"]}]}, "magic": "MS-DOS executable, PE for MS Windows (GUI) Intel 80386 32-bit", "last_analysis_stats": {"harmless": 0, "type-unsupported": 5, "suspicious": 0, "confirmed-timeout": 0, "timeout": 10, "failure": 0, "malicious": 46, "undetected": 14}, "last_analysis_results": {"Bkav": {"category": "malicious", "engine_name": "Bkav", "engine_version": "1.3.0.9899", "result": "W32.AIDetect.malware1", "method": "blacklist", "engine_update": "20210429"}, "Elastic": {"category": "malicious", "engine_name": "Elastic", "engine_version": "4.0.21", "result": "malicious (high confidence)", "method": "blacklist", "engine_update": "20210420"}, "MicroWorld-eScan": {"category": "malicious", "engine_name": "MicroWorld-eScan", "engine_version": "14.0.409.0", "result": "Gen:Heur.Mint.Zard.1", "method": "blacklist", "engine_update": "20210430"}, "FireEye": {"category": "malicious", "engine_name": "FireEye", "engine_version": "32.44.1.0", "result": "Generic.mg.22065d825f644512", "method": "blacklist", "engine_update": "20210430"}, "CAT-QuickHeal": {"category": "undetected", "engine_name": "CAT-QuickHeal", "engine_version": "14.00", "result": null, "method": "blacklist", "engine_update": "20210430"}, "McAfee": {"category": "timeout", "engine_name": "McAfee", "engine_version": "6.0.6.653", "result": null, "method": "blacklist", "engine_update": "20210430"}, "ALYac": {"category": "malicious", "engine_name": "ALYac", "engine_version": "1.1.3.1", "result": "Gen:Heur.Mint.Zard.1", "method": "blacklist", "engine_update": "20210430"}, "Malwarebytes": {"category": "malicious", "engine_name": "Malwarebytes", "engine_version": "4.2.2.27", "result": "RiskWare.Tool.CK", "method": "blacklist", "engine_update": "20210430"}, "Zillya": {"category": "malicious", "engine_name": "Zillya", "engine_version": "2.0.0.4354", "result": "Trojan.POSCardStealer.Win32.119", "method": "blacklist", "engine_update": "20210430"}, "SUPERAntiSpyware": {"category": "undetected", "engine_name": "SUPERAntiSpyware", "engine_version": "5.6.0.1032", "result": null, "method": "blacklist", "engine_update": "20210430"}, "Sangfor": {"category": "timeout", "engine_name": "Sangfor", "engine_version": "2.9.0.0", "result": null, "method": "blacklist", "engine_update": "20210416"}, "K7AntiVirus": {"category": "malicious", "engine_name": "K7AntiVirus", "engine_version": "11.180.37048", "result": "Trojan ( 00544ddf1 )", "method": "blacklist", "engine_update": "20210430"}, "Alibaba": {"category": "malicious", "engine_name": "Alibaba", "engine_version": "0.3.0.5", "result": "TrojanSpy:Win32/Alinaos.d24c2769", "method": "blacklist", "engine_update": "20190527"}, "K7GW": {"category": "malicious", "engine_name": "K7GW", "engine_version": "11.180.37048", "result": "Trojan ( 00544ddf1 )", "method": "blacklist", "engine_update": "20210430"}, "Cybereason": {"category": "malicious", "engine_name": "Cybereason", "engine_version": "1.2.449", "result": "malicious.25f644", "method": "blacklist", "engine_update": "20210330"}, "BitDefenderTheta": {"category": "malicious", "engine_name": "BitDefenderTheta", "engine_version": "7.2.37796.0", "result": "AI:Packer.F218F1801E", "method": "blacklist", "engine_update": "20210429"}, "Cyren": {"category": "malicious", "engine_name": "Cyren", "engine_version": "6.3.0.2", "result": "W32/Heuristic-162!Eldorado", "method": "blacklist", "engine_update": "20210430"}, "SymantecMobileInsight": {"category": "type-unsupported", "engine_name": "SymantecMobileInsight", "engine_version": "2.0", "result": null, "method": "blacklist", "engine_update": "20210126"}, "Symantec": {"category": "malicious", "engine_name": "Symantec", "engine_version": "1.14.0.0", "result": "Trojan.Gen.MBT", "method": "blacklist", "engine_update": "20210430"}, "ESET-NOD32": {"category": "malicious", "engine_name": "ESET-NOD32", "engine_version": "23218", "result": "a variant of Win32/Spy.POSCardStealer.D", "method": "blacklist", "engine_update": "20210430"}, "Baidu": {"category": "timeout", "engine_name": "Baidu", "engine_version": "1.0.0.2", "result": null, "method": "blacklist", "engine_update": "20190318"}, "APEX": {"category": "malicious", "engine_name": "APEX", "engine_version": "6.158", "result": "Malicious", "method": "blacklist", "engine_update": "20210428"}, "Paloalto": {"category": "malicious", "engine_name": "Paloalto", "engine_version": "1.0", "result": "generic.ml", "method": "blacklist", "engine_update": "20210430"}, "ClamAV": {"category": "malicious", "engine_name": "ClamAV", "engine_version": "0.103.2.0", "result": "Win.Trojan.Alina-4", "method": "blacklist", "engine_update": "20210429"}, "Kaspersky": {"category": "malicious", "engine_name": "Kaspersky", "engine_version": "21.0.1.45", "result": "UDS:Trojan.Win32.Generic", "method": "blacklist", "engine_update": "20210430"}, "BitDefender": {"category": "malicious", "engine_name": "BitDefender", "engine_version": "7.2", "result": "Gen:Heur.Mint.Zard.1", "method": "blacklist", "engine_update": "20210430"}, "NANO-Antivirus": {"category": "malicious", "engine_name": "NANO-Antivirus", "engine_version": "1.0.146.25279", "result": "Trojan.Win32.Banker1.ebnywb", "method": "blacklist", "engine_update": "20210430"}, "AegisLab": {"category": "undetected", "engine_name": "AegisLab", "engine_version": "4.2", "result": null, "method": "blacklist", "engine_update": "20210430"}, "Avast": {"category": "timeout", "engine_name": "Avast", "engine_version": "21.1.5827.0", "result": null, "method": "blacklist", "engine_update": "20210430"}, "Tencent": {"category": "malicious", "engine_name": "Tencent", "engine_version": "1.0.0.1", "result": "Win32.Trojan.Generic.Hmre", "method": "blacklist", "engine_update": "20210430"}, "Ad-Aware": {"category": "malicious", "engine_name": "Ad-Aware", "engine_version": "3.0.21.179", "result": "Gen:Heur.Mint.Zard.1", "method": "blacklist", "engine_update": "20210430"}, "Trustlook": {"category": "type-unsupported", "engine_name": "Trustlook", "engine_version": "1.0", "result": null, "method": "blacklist", "engine_update": "20210430"}, "Emsisoft": {"category": "timeout", "engine_name": "Emsisoft", "engine_version": "2018.12.0.1641", "result": null, "method": "blacklist", "engine_update": "20210430"}, "Comodo": {"category": "malicious", "engine_name": "Comodo", "engine_version": "33484", "result": "TrojWare.Win32.Patched.KSU@5t5qg6", "method": "blacklist", "engine_update": "20210429"}, "F-Secure": {"category": "malicious", "engine_name": "F-Secure", "engine_version": "12.0.86.52", "result": "Trojan.TR/Downloader.Gen", "method": "blacklist", "engine_update": "20210331"}, "DrWeb": {"category": "malicious", "engine_name": "DrWeb", "engine_version": "7.0.49.9080", "result": "Trojan.PWS.Banker1.8391", "method": "blacklist", "engine_update": "20210430"}, "VIPRE": {"category": "malicious", "engine_name": "VIPRE", "engine_version": "92204", "result": "Trojan.Win32.Generic!BT", "method": "blacklist", "engine_update": "20210430"}, "TrendMicro": {"category": "timeout", "engine_name": "TrendMicro", "engine_version": "11.0.0.1006", "result": null, "method": "blacklist", "engine_update": "20210330"}, "McAfee-GW-Edition": {"category": "malicious", "engine_name": "McAfee-GW-Edition", "engine_version": "v2019.1.2+3728", "result": "BehavesLike.Win32.Generic.kc", "method": "blacklist", "engine_update": "20210429"}, "Trapmine": {"category": "type-unsupported", "engine_name": "Trapmine", "engine_version": "3.5.0.1023", "result": null, "method": "blacklist", "engine_update": "20200727"}, "CMC": {"category": "undetected", "engine_name": "CMC", "engine_version": "2.10.2019.1", "result": null, "method": "blacklist", "engine_update": "20210327"}, "Sophos": {"category": "malicious", "engine_name": "Sophos", "engine_version": "1.0.2.0", "result": "ML/PE-A + Troj/Trackr-Gen", "method": "blacklist", "engine_update": "20210430"}, "SentinelOne": {"category": "malicious", "engine_name": "SentinelOne", "engine_version": "5.0.0.20", "result": "Static AI - Malicious PE", "method": "blacklist", "engine_update": "20210215"}, "Avast-Mobile": {"category": "type-unsupported", "engine_name": "Avast-Mobile", "engine_version": "210430-04", "result": null, "method": "blacklist", "engine_update": "20210430"}, "Jiangmin": {"category": "malicious", "engine_name": "Jiangmin", "engine_version": "16.0.100", "result": "Trojan/Generic.asqjf", "method": "blacklist", "engine_update": "20210429"}, "Webroot": {"category": "undetected", "engine_name": "Webroot", "engine_version": "1.0.0.403", "result": null, "method": "blacklist", "engine_update": "20210430"}, "Avira": {"category": "malicious", "engine_name": "Avira", "engine_version": "8.3.3.12", "result": "TR/Downloader.Gen", "method": "blacklist", "engine_update": "20210430"}, "MAX": {"category": "malicious", "engine_name": "MAX", "engine_version": "2019.9.16.1", "result": "malware (ai score=100)", "method": "blacklist", "engine_update": "20210430"}, "Antiy-AVL": {"category": "malicious", "engine_name": "Antiy-AVL", "engine_version": "3.0.0.1", "result": "Trojan/Win32.AGeneric", "method": "blacklist", "engine_update": "20210430"}, "Kingsoft": {"category": "undetected", "engine_name": "Kingsoft", "engine_version": "2017.9.26.565", "result": null, "method": "blacklist", "engine_update": "20210430"}, "Microsoft": {"category": "timeout", "engine_name": "Microsoft", "engine_version": "1.1.18100.5", "result": null, "method": "blacklist", "engine_update": "20210430"}, "Gridinsoft": {"category": "malicious", "engine_name": "Gridinsoft", "engine_version": "1.0.39.131", "result": "Malware.Win32.Pack.516!se", "method": "blacklist", "engine_update": "20210430"}, "Arcabit": {"category": "undetected", "engine_name": "Arcabit", "engine_version": "1.0.0.886", "result": null, "method": "blacklist", "engine_update": "20210430"}, "ViRobot": {"category": "undetected", "engine_name": "ViRobot", "engine_version": "2014.3.20.0", "result": null, "method": "blacklist", "engine_update": "20210430"}, "ZoneAlarm": {"category": "malicious", "engine_name": "ZoneAlarm", "engine_version": "1.0", "result": "HEUR:Trojan.Win32.Generic", "method": "blacklist", "engine_update": "20210430"}, "GData": {"category": "malicious", "engine_name": "GData", "engine_version": "A:25.29483B:27.22838", "result": "Gen:Heur.Mint.Zard.1", "method": "blacklist", "engine_update": "20210430"}, "Cynet": {"category": "malicious", "engine_name": "Cynet", "engine_version": "4.0.0.27", "result": "Malicious (score: 100)", "method": "blacklist", "engine_update": "20210430"}, "BitDefenderFalx": {"category": "type-unsupported", "engine_name": "BitDefenderFalx", "engine_version": "2.0.936", "result": null, "method": "blacklist", "engine_update": "20200916"}, "AhnLab-V3": {"category": "undetected", "engine_name": "AhnLab-V3", "engine_version": "3.20.0.10177", "result": null, "method": "blacklist", "engine_update": "20210430"}, "Acronis": {"category": "malicious", "engine_name": "Acronis", "engine_version": "1.1.1.81", "result": "suspicious", "method": "blacklist", "engine_update": "20210211"}, "VBA32": {"category": "malicious", "engine_name": "VBA32", "engine_version": "5.0.0", "result": "TrojanPSW.Banker", "method": "blacklist", "engine_update": "20210430"}, "TACHYON": {"category": "undetected", "engine_name": "TACHYON", "engine_version": "2021-04-30.02", "result": null, "method": "blacklist", "engine_update": "20210430"}, "Cylance": {"category": "timeout", "engine_name": "Cylance", "engine_version": "2.3.1.101", "result": null, "method": "blacklist", "engine_update": "20210430"}, "Zoner": {"category": "undetected", "engine_name": "Zoner", "engine_version": "0.0.0.0", "result": null, "method": "blacklist", "engine_update": "20210429"}, "TrendMicro-HouseCall": {"category": "malicious", "engine_name": "TrendMicro-HouseCall", "engine_version": "10.0.0.1040", "result": "Mal_Bits", "method": "blacklist", "engine_update": "20210430"}, "Rising": {"category": "malicious", "engine_name": "Rising", "engine_version": "25.0.0.26", "result": "Stealer.AlinaPOS!1.C5B3 (CLOUD)", "method": "blacklist", "engine_update": "20210430"}, "Yandex": {"category": "malicious", "engine_name": "Yandex", "engine_version": "5.5.2.24", "result": "Trojan.GenAsa!qXHBe5f1nPw", "method": "blacklist", "engine_update": "20210430"}, "Ikarus": {"category": "malicious", "engine_name": "Ikarus", "engine_version": "0.1.5.2", "result": "Trojan.Win32.Spy", "method": "blacklist", "engine_update": "20210430"}, "eGambit": {"category": "undetected", "engine_name": "eGambit", "engine_version": null, "result": null, "method": "blacklist", "engine_update": "20210430"}, "Fortinet": {"category": "malicious", "engine_name": "Fortinet", "engine_version": "6.2.142.0", "result": "W32/Spy.POSCARDSTEALER.D!tr", "method": "blacklist", "engine_update": "20210430"}, "MaxSecure": {"category": "timeout", "engine_name": "MaxSecure", "engine_version": "1.0.0.1", "result": null, "method": "blacklist", "engine_update": "20210430"}, "AVG": {"category": "timeout", "engine_name": "AVG", "engine_version": "21.1.5827.0", "result": null, "method": "blacklist", "engine_update": "20210430"}, "Panda": {"category": "undetected", "engine_name": "Panda", "engine_version": "4.6.4.2", "result": null, "method": "blacklist", "engine_update": "20210429"}, "CrowdStrike": {"category": "malicious", "engine_name": "CrowdStrike", "engine_version": "1.0", "result": "win/malicious_confidence_100% (W)", "method": "blacklist", "engine_update": "20210203"}, "Qihoo-360": {"category": "undetected", "engine_name": "Qihoo-360", "engine_version": "1.0.0.1120", "result": null, "method": "blacklist", "engine_update": "20210430"}}, "reputation": 0}, "type": "file", "id": "bc3df8753583d38f479b6151a106b9acaecf83f209ca01774acbcd4132da1572", "links": {"self": "https://www.virustotal.com/api/v3/files/bc3df8753583d38f479b6151a106b9acaecf83f209ca01774acbcd4132da1572"}}], "links": {"self": "https://www.virustotal.com/api/v3/search?query=bc3df8753583d38f479b6151a106b9acaecf83f209ca01774acbcd4132da1572"}}

Had to find a short one :D

Thanks in advance!

malicialab commented 2 years ago

This totally fell in a void. Apologies for that. The issue was that 'attributes' is supposed to be a dictionary rather than a list of a single dictionary entry where a dictionary was expected, not a list. Not sure where the JSON came from, but we have not heard this issue from other users. I am closing the issue and apologizing again for the lack of answer.