MISP not receiving any attributes

[icepaule]

Hello world,

does anybody have the same issue and found a solution for it? Cuckoo ist comntacting and sending the analysis to my MISP, but all the relevent data of the attributes are not uploaded to it. Just as if upload iocs were "no".

Cuckoo shows: [modules.reporting.misp] ERROR: Failed to generate JSON report: 'NoneType' object has no attribute 'get'

Any help would be highly appreciated. :-)

Thanks a lot

Cheers Marcus

[misp] enabled = yes apikey = xxxxxx url = https://misp.xxxx.de min_malscore = 0 threads = 5 extend_context = yes upload_iocs = yes distribution = 0 threat_level_id = 2 analysis = 2 title = IOCs from the SOC cuckoo analysis: network = yes ids_files = yes dropped = yes registry = yes mutexes = yes

Log from MISP to prove the connection is established: [2017-03-15 17:35:27] main.INFO: got {"queue":"email","id":"fb6489befe3fa9ba068b834b7a20c324","class":"EventShell","args":[["alertemail","5","4080","3837",null]]} {"type":"got","args":"[object] (Resque_Job: {\"queue\":\"email\",\"id\":\"fb6489befe3fa9ba068b834b7a20c324\",\"class\":\"EventShell\",\"args\":[[\"alertemail\",\"5\",\"4080\",\"3837\",null]]})","worker":"OSINT:2092"} [] [2017-03-15 17:35:27] main.INFO: Processing ID:fb6489befe3fa9ba068b834b7a20c324 in email {"type":"process","worker":"OSINT:2092","job_id":"fb6489befe3fa9ba068b834b7a20c324"} [] [2017-03-15 17:35:28] main.INFO: done ID:fb6489befe3fa9ba068b834b7a20c324 {"type":"done","job_id":"fb6489befe3fa9ba068b834b7a20c324","time":868,"worker":"OSINT:2092"} []

[doomedraven]

Hm no, but i would suggest check misp module and add some logging before and after all gets to verify where exactly it fails, once i will be with pc i will check it too

[icepaule]

Thanks doomedraven, I checked the whole day actually, but could not find a real hint to the problem. Even in debug mode nothing more then that one message was seen. All modules seem to be installed but still that JSON problem is driving me mad. :-)

I'll keep digging. Thanks a lot for sharing this issue with me.

[doomedraven]

Can you share misp server version, pymismp version and sample or url which generates fail?

[doomedraven]

also to identify where it file replace that try https://github.com/spender-sandbox/cuckoo-modified/blob/master/modules/reporting/misp.py#L218 with if True and remove at the bottom

        except Exception as e:
            log.error("Failed to generate JSON report: %s" % e)

and reexecute that will help where it fails on your side, let me know where, to fix it

[icepaule]

Hi there, pymisp is version 2.9.1 The URL send to cuckoo for analysis and send to MISP was: http://gdyhousingservices.com/zzz.exe (potential malware, take care) My MISP ist version 2.4.66

Sorry, I don't get your last comment with "if true" (I'm not verry well into python)

Thx for all the help.

Marcus

[doomedraven]

Well till tomorrow i wont be with pc, so if you want try solve that before, go to that file in your local cuckoo to the same line and replace it

[icepaule]

OK, just did that and seems to work. ;-) The output that was given is pasted below. Lookes like line 137 in cuckoo2misp. I'll have a deeper look into it. Maybe I find something out myself.

Anyway, thanks a lot for your help.

Marcus

2017-03-16 06:59:55,715 [lib.cuckoo.core.plugins] ERROR: Failed to run the reporting module "MISP": Traceback (most recent call last): File "/opt/cuckoo/lib/cuckoo/core/plugins.py", line 631, in process current.run(self.results) File "/opt/cuckoo/modules/reporting/misp.py", line 262, in run self.cuckoo2misp(results, whitelist) File "/opt/cuckoo/modules/reporting/misp.py", line 137, in cuckoo2misp filename=results.get('target').get('file').get('name'), AttributeError: 'NoneType' object has no attribute 'get'

[doomedraven]

Replace

filename=results.get('target').get('file').get('name'),

To filename=results.get('target',{}).get('file',{}).get('name', ''),

This will fix your issue as that url and not file, i will push fix later

[icepaule]

Unbelievable, I fixed it.... Attributed are now showing up perfectly in my MISP. Below are the lines I've changed.

But please have a look what I did there and if it's correct. As I told you, I'm not in Python that much. - But anyway, data arrives at the MISP now. -I'm so happy-

Thanks so much for helping me to help myself. Well done.

self.misp.add_hashes(event, category='Payload delivery', filename=results.get('target', {}).get('file', {}).get('name', {}), md5=results.get('target', {}).get('file', {}).get('md5', {}), sha1=results.get('target', {}).get('file', {}).get('sha1', {}), sha256=results.get('target', {}).get('file', {}).get('sha256', {}), ssdeep=results.get('target', {}).get('file', {}).get('ssdeep', {}), comment='File: {} uploaded to cuckoo'.format(results.get('target', {}).get('file', {}).get('name', {})))

[doomedraven]

Hehe i glad it work, i didnt test it with urls analysis but i will push to fix it, so maybe time to close it? ;)

[doomedraven]

Can you test this fix? https://github.com/spender-sandbox/cuckoo-modified/pull/426

[icepaule]

Well, sorry, but not quite. With your version of the change I get one empty MISP entry and one with the correct IOC's. - If it helps, the version I did, gives only one entry to the MISP.

Cheers Marcus

[doomedraven]

Hm weird, i will check

[doomedraven]

that is so weird, can you do few more tests? as that check works for one small block which exists only when file is analyzed and not url

[icepaule]

sure, I'll do what ever you want me to... :-)

[doomedraven]

can you analyse ipinfo.io? i can't test it as we doing changes in our server and we took down misp, but i checked code and that not make sense if that generates empty event, there should be some fail or something because that if looks fine, do analysis plz of few more urls and files and let me know