Cuckoo-Spender fork not compatible with threat_note.

[big-gunter]

Hi all,

Hopefully this is not a dumb error - I'm having an issue with Cuckoo integration. When I go to import data I receive the following error:

(pysqlite2.dbapi2.InterfaceError) Error binding parameter 0 - probably unsupported type. [SQL: u'SELECT indicators._id AS indicators__id, indicators.object AS indicators_object, indicators.type AS indicators_type, indicators.firstseen AS indicators_firstseen, indicators.lastseen AS indicators_lastseen, indicators.diamondmodel AS indicators_diamondmodel, indicators.campaign AS indicators_campaign, indicators.confidence AS indicators_confidence, indicators.comments AS indicators_comments, indicators.tags AS indicators_tags, indicators.relationships AS indicators_relationships \nFROM indicators \nWHERE indicators.object = ?\n LIMIT ? OFFSET ?'] [parameters: ({u'country_name': u'unknown', u'ip': u'91.230.211.139', u'inaddrarpa': u'', u'hostname': u''}, 1, 0)]

I can see the successful get requests in the API window from cuckoo (utils/python api.py -H 0.0.0.0), when I initiate the request through threat_note, but then am met with that error.

I reverted the Cuckoo database back to SQlite (was running MySQL), however this made no difference (other than Cuckoo throwing up a warning that it doesn't play nice with SQlite). I reverted everything back to default ports (assuming it was a configuration error on my behalf) - but still the same error.

Threat_note can see new analysis' however all result in the same error.

As I said though, I am far from a SQlite/MySQL guru, this could well be my error, however everything else is functioning perfectly in all configuration formats (SQlite, MySQL, elasticsearch talking to MySQL etc).

Help!

[big-gunter]

I have continued playing around with this and have noted that when using docker in lieu of honcho I get a similar error when attempting to talk to Cuckoo.

(sqlite3.InterfaceError) Error binding parameter 0 - probably unsupported type. [SQL: u'SELECT indicators._id AS indicators__id, indicators.object AS indicators_object, indicators.type AS indicators_type, indicators.firstseen AS indicators_firstseen, indicators.lastseen AS indicators_lastseen, indicators.diamondmodel AS indicators_diamondmodel, indicators.campaign AS indicators_campaign, indicators.confidence AS indicators_confidence, indicators.comments AS indicators_comments, indicators.tags AS indicators_tags, indicators.relationships AS indicators_relationships \nFROM indicators \nWHERE indicators.object = ?\n LIMIT ? OFFSET ?'] [parameters: ({u'country_name': u'unknown', u'ip': u'8.8.8.8', u'inaddrarpa': u'', u'hostname': u''}, 1, 0)]

[big-gunter]

Further testing has shown it only appears to be failing with the modified, hardened, Spender fork of Cuckoo. I have managed to get it to integrate with the current Cuckoo 2.0-rc1.

Unfortunately my level of knowledge tops out at working out when rather than what.

It is unfortunate as the modified fork is an excellent tool in dealing with vm-aware malware. If anyone is working on fixing this I would love to know!

I have left this comment in case anyone else comes searching for an answer.

[k3vb0t]

I haven't tried the cuckoo version(w/ TN) yet. I am a) sudo proficient with cuckoo, and b) have not tried to tie these 2 together yet. I do know, however, that the original cuckoo updated a ~couple months ago. If you are using a modified fork from any of the other repos, there is a HIGH probability something will break. Can you link me to your exact version of cuckoo you are trying to use? I'll try and rep the issue with it in the next week or so. Thanks.

[big-gunter]

Thanks for getting back to me! I tried to test out multiple versions of Cuckoo today. While their github is delivering the dev version (and I haven't tested that), the current tarball (and stable version) from their website is RC1 and I have manged to get that to work ( I updated my post after the initial when I confirmed it was actually working with RC1).

Indeed my issue was that I am attempting to use it with hardened Spender fork located here: https://github.com/spender-sandbox/cuckoo-modified

[k3vb0t]

Have you tried this version yet by chance https://github.com/brad-accuvant/cuckoo-modified ? Just wondering. I figure I'll step through them all along the way and it would be cool to compare notes.

[big-gunter]

Hi! The accuvant fork is no longer maintained. Interestingly enough, the Spender fork is actually the continuation of the accuvant fork as of December 2015.

[k3vb0t]

I know, I'm a contributor to the Accuvant fork(in a very small way) :) , I was just wondering what fork the issues really started in and was going to start back-tracing from there.

[big-gunter]

Oh cool! Sorry for my ignorance! My days just about to start (6AM), so I'll clone the accuvant fork and post up the results here during the day!

[big-gunter]

Alrighty, cloned and configured the final version of the Accuvant fork. I left the DB as SQLite and then tried again as MySQL had tried this prior but just trying to rule out DB issues on Cuckoos side). Both resolve in the same error in my very first post (pysqlite2.dbapi2.InterfaceError) .

[k3vb0t]

Cool, thanks for doing that. I'm in the process of building up a new box from scratch so I should be working with this pretty soon. I'll let you know what, if anything, I can find/solve.

[big-gunter]

That would be awesome, thanks so much!

[k3vb0t]

I'm able to reproduce this error getting (sqlite3.InterfaceError). I'll start messing around with it and see if I can figure it out.

[k3vb0t]

Pretty sure the issue is right around here: https://github.com/defpoint/threat_note/blob/master/threat_note/libs/cuckoo.py#L21 I'll have to tinker around with it some more to figure out exactly what it is.

[k3vb0t]

Ok, I have this (mostly) fixed. I need to do a bit more work on it and then I'll push out a change.

[k3vb0t]

I pushed out the fixes, all it seems to have ended up being was this: https://github.com/defpoint/threat_note/pull/163/commits/651d7146e22054c449e6c37e7112baceeb0b8935#diff-4d7536e59682670f8c4b08b61236c9baR358

[big-gunter]

Awesome work - can confirm is now working with the hardened Spender fork! Thanks so much!

[k3vb0t]

yw, only took me ~5 hours of reading to add one itty bitty line of code :)

I didn't test this on the original cuckoo version, but I think ppl should be using this fork anyway

[big-gunter]

I use both, the difference in results is amazing - things classified as benign by the standard fork vs classifications from the spender fork and then the spender fork utilising VMCloak created VMs. The later two are close if not the same.

Once again, I appreciate the effort - an awesome bit of kit.

[k3vb0t]

I had not tried VMCloak yet, thanks for the tip :) Feel free to close this issue if you feel it's been resolved