elastic / integrations

Elastic Integrations
https://www.elastic.co/integrations
Other
194 stars 421 forks source link

Change Rapid7 InsightVM integration to one doc per vulnerability #9354

Open jamiehynds opened 6 months ago

jamiehynds commented 6 months ago

Similar to our recent update to Qualys VMDR update, we have similar feedback for our Rapid7 InisghtVM integration. Can we make a similar change to the Rapid7 integration, based on the user feedback as follows:

In our pursuit of refining our security posture through enhanced data analysis, the integration between Elastic and Rapid7 has surfaced areas ripe for optimization, particularly concerning the management of vulnerabilities data. Presently, the nightly indexing process for vulnerabilities not only encompasses all modified vulnerabilities but also tends to re-index vulnerabilities indiscriminately. This includes a broad spectrum of vulnerabilities scanned by Rapid7, rather than focusing solely on those relevant to our specific environment. Such an approach results in considerable data redundancy and inefficiency, straining our resources.

To address this, we propose the introduction of a new data category within Elastic, designated as 'state-*'. This category would advocate for the use of monolithic indices that leverage non-random document identifiers (not based on a timestamp), such as the unique ID of each vulnerability. Implementing this change would ensure that each vulnerability is represented by a single document within the dataset, markedly reducing the volume of data processed and stored daily. This adjustment would not only streamline the handling of vulnerability data but also significantly enhance the utility and performance of Rapid7 integrations within the Elastic ecosystem. We are keen to discuss this proposal further and explore the possibility of its implementation, hopeful that Elastic sees the value in optimizing the management of vulnerability data for mutual benefit.

elasticmachine commented 6 months ago

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

mbudge commented 4 months ago

Will there be a document for each vulnerability per host and service or is it one document per vulnerability across all hosts/services?

iukea1 commented 3 months ago

Can't wait