Add osquery and Fleet Desktop logs to `fleetd_logs` table

[dantecatalfamo]

Problem

Currently the fleetd_logs table only collects logs from orbit. It would be nice if we could also collect logs from fleet-desktop as well, so the logs can be collected in one place.

fleetd_logs are currently collected in-memory and in-process. fleet-desktop runs as a separate child process, so its logs are not included in the collection.

Potential solutions

Output the logs from fleet-desktop in JSON format, collect them through a pipe in the parent process, parse them just like we parse the paren't JSON logs, and include them in the log table.

It would also be nice to add a field to the logs table to indicate that the entries are from the desktop component.

[noahtalerman]

It would be nice if we could also collect logs from orbit-desktop as well, so the logs can be collected in one place.

Hey @dantecatalfamo can we call this new column fleet-desktop in the table instead? This way, we'd be consistent w/ the product and docs.

For example, check out the Host details page:

[noahtalerman]

Pulling this one off the feature fest board but leaving ~engineering-initiated so it can go through the eng-initiated prioritization process: https://fleetdm.com/handbook/engineering#create-an-engineering-initiated-story

FYI @lukeheath

[dantecatalfamo]

Hey @noahtalerman, we definitely can. We should probably rename the existing table to orbit_logs then

[noahtalerman]

Could we expand the existing fleetd_logs table to collect all fleetd logs? Orbit and Fleet Desktop (now) + osquery (later)

[noahtalerman]

@dantecatalfamo forgot to tag you ^^

[dantecatalfamo]

@noahtalerman For sure! That's what I was initially thinking. We could add an extra column to specify which component the logs come from

[lukeheath]

@georgekarrv @dantecatalfamo I am prioritizing this to the drafting board for estimation.

[georgekarrv]

Hey team! Please add your planning poker estimate with Zenhub @dantecatalfamo @ghernandez345 @gillespi314 @mna @roperzh

[nonpunctual]

related?

• https://github.com/fleetdm/fleet/issues/290
• https://github.com/fleetdm/confidential/issues/5902
• https://fleetdm.com/tables/fleetd_logs

@noahtalerman @marko-lisica expanding the collection capabilities for collecting logs on the Host (client-side) will fulfill customer requests around fleetd monitoring as mentioned in 5902 (in confidential repo) referenced above. Most important would be collecting imo would be collecting the "last_seen" time or client-side equivalent. Even if there is a cart / horse, chicken / egg, whatever problem, allowing admins to access fleetd, Fleet Desktop state from Fleet console would allow admins to satisfy SOC2-type reporting requirements for installed agents like Fleet.