intelowlproject / IntelOwl

IntelOwl: manage your Threat Intelligence at scale
https://intelowlproject.github.io
GNU Affero General Public License v3.0
3.82k stars 432 forks source link

XForce Intelligence Analyzer #139

Closed bowling closed 3 years ago

bowling commented 4 years ago

Hello,

Is it possible to add XForce Intelligence (IBM XForce) to the list of APIs that can be used? They have a fully documented API for public use, along with limited free or paid API access. Some of the more useful ones (such as Botnet C2 categories) are not metered.

https://api.xforce.ibmcloud.com/doc/ https://exchange.xforce.ibmcloud.com/ https://exchange.xforce.bmcloud.com/settings/api

"All" it requires is a valid API key and password, plus a valid GET request with a standardized format for the data. Example:

(Botnet C2 Servers - IPv4) curl https://api.xforce.ibmcloud.com/xfti/bots/ipv4 { "FeedCategory": "Bots", "FeedType": "IPv4", "Version": "0000000010", "CreationDate": "2020-08-10T07:26:00.000Z", "IndicatorCount: "2", "data": ["127.0.0.1", "..."] }

mlodic commented 4 years ago

Hi, thank you for your interest in the project!

Some IBM API endpoints cannot be implemented. One important thing to remember is that IntelOwl is a tool to enrich data. This means that users should start with an observable (IP,domain,...) or a file that they would like to analyze and ask IntelOwl to process it. For instance, the endpoint you mentioned /xfti/bots/ipv4 cannot be integrated because is a generic threat intelligence information gathering and users do not need to specify any observable to extract this data. Otherwise, there are other API endpoints that could be leveraged:

For IP addresses:

For malware hashes:

For URL/domains:

Implementing these API endpoints in a specific analyzer is easy and straightforward also for new contributors. You just need to follow the instructions plus some python code experience. I suggest every interested reader to check the documentation and try to implement this. Feel free to ask for questions if needed

Also, to leverage the endpoints like the one you mentioned, we can also implement a more complex solution. For instance, we could periodically extract the list of Bots IP addresses from https://api.xforce.ibmcloud.com/xfti/bots/ipv4 and save these data internally in the application, as a simple file. Then, when a user analyzes an IP, a specific analyzer could try to find that IP in this list and show the result in the case it is found. At the moment, a very similar implementation is done for the Tor analyzer. Obviously, a requirement is that the endpoint where we download the data periodically is free to use.

bowling commented 4 years ago

Hey,

Thank you! I'll look into adding a custom analyzer in for this. If I can get it too work properly I'll send it off to you for review (and potentially addition).

Cheers, Mike

sp35 commented 3 years ago

@mlodic I would like to work on this. I need to create a single analyzer XForce Intelligence Analyzer which would support the observables ip, url, and hash by using the endpoints quoted below.

Since we got multiple endpoints for ip and url analysis, I would have to call all of them for an analysis request.

For IP addresses:

For malware hashes:

For URL/domains:

mlodic commented 3 years ago

Sure! You have been assigned!

sp35 commented 3 years ago

@mlodic Can you please close this issue? I'm not sure why it didn't get closed, the PR was linked though.

mlodic commented 3 years ago

This should work once we merge that PR to the default branch (master). Anyway I can do it now