[Cloud Security] POC sharing Cloud Security APIs

[JordanSh]

Summary:

as a part of https://github.com/elastic/security-team/issues/9015 and https://github.com/elastic/security-team/issues/9137 we need to show components related to CSP in the context of security_solution plugin. Specifically, we have:

• Misconfiguration and Vulnerability posture preview per host.name and user.name
• Misconfiguration and Vulnerability basic data grid with findings per host.name and user.name

Specific data that we are concerned for these epics:

• status of the data streams/indexes and integrations. Basically, the information which is already covered by CSP's status/ API. We also might need to cover 3rd party integrations and data stream/index statuses there
• Current Misconfiguration and Vulnerability posture (number of failed/passed findings, number of vulnerabilities split by severity)
• Misconfiguration and Vulnerability finding documents for a basic data grid (with pagination, sorting, filtering)

In this issue API means any programmatic interface that can be shared: REST APIs over HTTP, services encapsulating data fetching, set of hooks or other components shared across plugins, etc.

The strategic questions we want to answer in this ticket:

1. how do we expose our data fetching APIs so that we reuse them across security_solution and cloud_security_posture plugins? The ideal state is that we reduce the number of places to update to a minimum. Situation to avoid: disconnected code path across multiple plugins doing things differently
2. how to avoid problems with cyclic dependencies so that we could freely use those APIs in any plugin, or at minimum in both security_solution and cloud_security_posture plugins. See related PR where this challenge surfaced and explored for PLI component

More specific questions to answer:

1. How do we encapsulate our REST APIs to be used in different plugins? Can we create a service (maybe as a separate package) to be used in both security_soltuion and cloud_security_posture plugin
2. Does it make sense to reuse the Data Grid from our Findings page in the flows of Security Solution? Is it feasible as on the Findings page we use the data grid with grouping which has many more features than we plan for the data grid on the flyouts?
3. If we don't want to reuse the e2e data grid component, what parts we can share between data grids, eg. Data View data fetching logic

Definition of Done:

• [ ] Verify that the status API and latest findings API can be successfully shared between Kibana plugins
• [ ] Obtain approval from stakeholders to ensure the task meets acceptance criteria before closing this ticket
• [ ] Optionally, share the table component to display the data in another plugin, if this proves to be complex, it can be done as another poc ticket

[elasticmachine]

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

[maxcold]

@JordanSh Can you elaborate more on the need for exposing our APIs? I was thinking if we could use the existing Security Data View to query this date, but not sure how the flyout works exactly and if that's feasible

[JordanSh]

@JordanSh Can you elaborate more on the need for exposing our APIs? I was thinking if we could use the existing Security Data View to query this date, but not sure how the flyout works exactly and if that's feasible

The idea was to use our latest findings APIs instead of manually querying the data view. Since the Alerts detail flyout suppose to display the latest findings table, filtered by the shown entity, my assumption was that we would have the exact same fetching for both the alerts findings table, and our findings table.

Meaning we could reuse our own APIs which are already tested and integrate with our table component (which we would like to explore the option of sharing it as well). This will save us the trouble of maintaining and testing duplicate code.

The main problem I anticipate is cyclic dependencies, though there are ways around it. I thought that making a POC and understanding the pros and cons for this approach is needed

[JordanSh]

ticket refactor looks great @maxcold. R Regarding cyclic dependencies, of course I trust you to do your own research, but if you want you can also take a look at this PR which i ended up closing because i managed to avoid a solution which revolved around cyclic deps, but it might be valuable for your research