Software inventory and certificate authorities on Host details page

[noahtalerman]

This issue tracks the addition of software inventory and certificate authority information on the Hosts details page in the Fleet UI. Each of these items will be presented in the UI within the context of a singular host.

• Surface applications and packages as well as the respective versions of each
• Surface certificate authorities

[pixelsquared]

Some ideas I have had:

• Map applications to CPE
• Perhaps a query to identify and get the version number for each app maybe in a different repo
• The ability to check the CPEs for known vulnerabilities

What are your thoughts?

[noahtalerman]

Hi @pixelsquared, thank you for the ideas!

Map applications to CPE

The CPE dictionary is a new data source for me. Can you help me understand why it would be helpful to map applications to CPE? Is the ultimate goal related to just your 3rd bullet, or are there use cases for CPE mapping other than the ability to check for known vulnerabilities?

Perhaps a query to identify and get the version number for each app maybe in a different repo

We're planning on including the versions along with the application/package name. I've edited this issue description to reflect this plan.

Would it be helpful to reveal the query that's used to get the version number? And what exactly do you mean by "in a different repo?"

[pixelsquared]

The main reason for mapping to CPE is the ability to check for known vulnerabilities.

Let me explain a bit some issues I have had in the past when trying to create a list of all install app as CPE. On windows some applications will not show their version number in the installed programs list or not even show up there such as a installed web application. Also some applications might only show one installed entry but are actually multiple applications with different CPE. And on Linux some application will install without a DEB or RPM and have their own installer. So just doing SELECT * FROM programs; or similar will not get enough data to build a list of all the CPE installed.

So what I am suggesting is creating a database of queries and configuration for gathering applications and their CPE.

[pixelsquared]

This might need to be a separate issue because this issue is about the UI page in the host details or do you want to keep this here?

[noahtalerman]

Let's move this discussion to the Vulnerability management issue #405

[noahtalerman]

Update from Monday 4.12.2021

Goal

Place software inventory behind a feature flag in the Fleet 3.11.0 release (scheduled 4.21.2021)

There are MySQL performance concerns with storing installed software data for all hosts in a Fleet instance. Releasing software inventory behind a feature flag will allow us to ship the feature quickly while warning adopters of potential performance concerns.

How?

• [x] Prepare Fleet UI to consume new software data. UI mockups are here on Figma
  • [x] @noahtalerman determines which table option already present in the frontend will be best to replicate for the Software table on the "Host details" page
    • Let's use the QueriesList.
  • [x] Frontend PRs with dummy data
    • The feature flag will be dictated by the backend. The currently discussed frontend solution is to not render the Software table if there is no software subkey returned by the endpoint.
• [x] Backend changes are made to support changes to the api/v1/fleet/hosts/{id} endpoint. API wireframes are here on Figma
  • These query wireframes provide a jumping off point for collecting the necessary software data from hosts:
  • macOS query
  • Windows query
  • Linux query
  • FreeBSD query
  • [x] Backend PRs with changes to the above endpoint

cc @zwass

[noahtalerman]

Update on releasing Software inventory

These two items will be included in the release of Fleet 3.11.0. Now scheduled for Wed April 28, 2021:

• The software property in the api/v1/fleet/hosts/{id} endpoint
• The "Software" table on the Host details page

Frontend changes

Frontend changes are included in this draft PR: #643

API changes

• The improvements to the Fleet API include adding the name, version, and type properties to the api/v1/fleet/hosts/{id} API endpoint.
• The query wireframes are updated to support the additional properties
  • macOS query
  • Windows query
  • Linux query
  • FreeBSD query

API wireframes are included in this draft PR: #669

API wireframe:

Request: GET api/v1/fleet/hosts/1

Response

"host": {
  ...
  "software": [
    {
      "name": "CentOS Linux $releasever - AppStream",
      "type": "Package (YUM)",
      "version": "",
    },
    {
      "name": "curl",
      "type": "Package (RPM)",
      "version": "7.61.1",
    },
  ]
}

[noahtalerman]

The release of Fleet 3.11.0 includes:

• The software property in the api/v1/fleet/hosts/{id} endpoint
• The "Software" table on the Host details page

In 3.11.0, the software inventory is placed behind a feature flag. Fleet's feature flag documentation includes instructions on how to enable the software inventory feature.

This issue will continue to track updates and improvements to the software inventory feature as well as track the task of adding certificate authorities to the Host details page.