certtools / intelmq

IntelMQ is a solution for IT security teams for collecting and processing security feeds using a message queuing protocol.
https://docs.intelmq.org/latest/
GNU Affero General Public License v3.0
966 stars 295 forks source link

Added Bot Support to Sucuri Audit Trails #1904

Open MarcosxDeveloper opened 3 years ago

MarcosxDeveloper commented 3 years ago

Hello!

I made a Python Script for get IoCs of our Web Application Firewall [in this case Sucuri] and send to the organization SIEM.

Currently in my organization we are migrating all the tools so that everything enters through intelmq.

so I would like to ask your help to migrate my tool to intelmq bots.

Sucuri Example:

ghost commented 3 years ago

Hi,

I'd split this into two problems, the sucuri input and the output to SIEM. And first, I propose to focus on the first. Your second link says, that there's an API for downloading the Audit Trails and the URL is https://waf.sucuri.net/api?v2&k=$API_KEY&s=$API_SECRET&a=audit_trails&date=${1}, so a HTTP collector seems fine here. For the date you make make use of the http_url_formatting parameter (see https://intelmq.readthedocs.io/en/latest/user/bots.html#generic-url-fetcher and the two examples https://intelmq.readthedocs.io/en/latest/user/feeds.html#cz-nic ). Do you also need to deal with limit and offset?

If you have a working collector bot configuration, we'd of course appreciate a snippet for the feed configuration so it gets listed on https://intelmq.readthedocs.io/en/latest/user/feeds.html

Next steps are then parsing, eventually de-duplicating and outputting to SIEM

MarcosxDeveloper commented 3 years ago

Hi Wagner

I already tried to use the Sucuri API using only the http collector. but as advanced logic is needed to get all the events, it is necessary to make a custom bot where the pagination parameters are handled.

to complete the integration with Sucuri I think it only necessary to make the collector (inheriting from URL fetcher) and a parser (inheriting from JSON parsers)

Regards

ghost commented 3 years ago

I already tried to use the Sucuri API using only the http collector. but as advanced logic is needed to get all the events, it is necessary to make a custom bot where the pagination parameters are handled.

If it's only about pagination, I think we can implement this in the generic http collector as well. That gives the advantage that we can use all the other existing features (including it's well-tested error-handling).

to complete the integration with Sucuri I think it only necessary to make the collector (inheriting from URL fetcher) and a parser (inheriting from JSON parsers)

Do you know of a public example of the data somewhere?

MarcosxDeveloper commented 3 years ago

you're right,

I thought about adding that paging functionality to the URL Fetcher, but personally I think that would only work in some cases.

You can see an example of the Sucuri Web API here:

https://sucuri.net/demo/firewall/settings-api/#audit-trails