search

intelowlproject / IntelOwl

IntelOwl: manage your Threat Intelligence at scale
https://intelowlproject.github.io
GNU Affero General Public License v3.0
3.84k stars 439 forks source link

Results report #816

Closed serpaldom closed 2 years ago

serpaldom commented 2 years ago

Hello to all of you I would like to ask if there is a way to visualize the data, provided by the different analyzers configured in InteOwl, in a more visual or comfortable way, that I don't know at the moment.

Thank you very much

Best regards!

mlodic commented 2 years ago

At the moment there is no way except the JSON visualization. (Well there is the ElasticSearch integration with the Kibana GUI but I do not think this is what you are looking for because you would look at the data outside from the actual web application.)

IntelOwl saves you the time to retrieve all these data from different sources/tools but then an analyst or an integration with a TIP (Threat Intel Platform) is required to sort out, aggregate and make sense of the data.

I am interested in understanding what would you expect in terms of visualization. Right now we are rewriting the GUI, so it is a good time to get any idea from the users. Thanks

serpaldom commented 2 years ago

Hi mlodic First of all, happy new year and best wishes for this year.

I think it would be a good idea to include the following, where possible:

  1. Scanning several observables in the same job. (Example: scan 4 differents IP like 8.8.8.8,4.4.4.4,3.3.3.3,2.2.2.2)
  2. Possibility of previewing, within a job, in the "Analyzers reports" section, the result of an API. For example, categorise as: Malicious, non-malicious, error... the result obtained, depending on the data received from the api. Perhaps a percentage estimate of whether it is malicious or not could be applied depending on whether the report is very old, is recent, has been reported several times...
  3. Possibility of generating a visual PDF report that exposes the different data obtained, in a more visual way. Based in Python-Django, maybe this library could help to this: pdfkit (https://pypi.org/project/pdfkit/)
  4. Possibility of exporting all the json data from a job to a downloadable file.

Thanks for all! Best!

mlodic commented 2 years ago

thanks for the wishes! same to you!

About your suggestions:

1- we have an opened issue regarding that (#732). We will implement this soon. 2- this would require the application of some proprietary threat intelligence logic. And this should be optional because some results could be "biased". I mean, for your use case you could find relevant some information, but for other cases that information could be not useful. So it is really difficult to perform a generic "evaluation" and "aggregation" of the results of a job. This is why we are planning to add some connectors that will perform this task based on different use cases. However I am afraid it won't be open source but this will be probably proposed in the future IntelOwl-as-a-service application we will provide. 3- PDFs can also be great for exporting results from the platform. We will evaluate this addition 4- the JSON data can already be exported manually. If you click on the button "<>" in the job result view: image You can then view the JSON page then, if you are on Firefox, right click, choose "Save page" and export it. The same should work on Chrome with the JSON viewer extension. Anyway, yes, we could add an easy button to facilitate this operation and maybe provide different options with different formats.

I'll create some issues regarding these points. Thank you

mlodic commented 2 years ago

EDIT: I have added point 2 that was only a draft

kx499 commented 2 years ago

I just started playing with the project and I was actually wondering something similar - if there was a way visualize the # of meaningful results returned and highlight the ones you may want to focus on first. I get the bias comments, but maybe gear it more towards highlighting actionable data and less around a verdict on maliciousness. I've seen something like this done with standard summary section in the json other places - not sure if its possible here. image

mlodic commented 2 years ago

Thank you for your suggestion.

Yeah, if I am correct that is a screen from Cortex.

If I think about this specific example, it makes completely no sense to me. Only because there are records in AbuseIPDB would it mean that an observable is malicious? And if there are not is it safe?

I mean, all threat analysts could agree that this is completely wrong and it is an over simplification of the problem. This is also dangerous because it could lead people to wrong conclusions.

Understanding, analyzing, aggregating and evaluating the data is a really complex process that cannot be solved with just 2 lines of code. And it is extremely subjective to each one experience or use case. So you need to find your own meaningful results. Otherwise, as I mentioned, you can leave that job to external and "biased" analysts or integrations that can do it for you but you need to trust them.

And yeah, right now IntelOwl is an "extractor" of data, not a real Threat Intelligence Platform. But we want it to become a TIP. This is why we are about to integrate tools that can be used to perform investigations and analysis of the data inside IntelOwl: