OS settings not reaching `Verified` status

[sabrinabuckets]

Fleet version: (head to the "My account" page in the Fleet UI or run fleetctl --version) fleetctl - version fleetd-chrome-v1.1.3-beta-79-ge7d2aee8a-dirty branch: main revision: e7d2aee8a1df17d58c8f89ba912cbd21a7e671ae build date: 2024-02-08 build user: bri go version: go1.21.3

Operating system: (e.g. macOS 11.2.3) Affects macOS & Windows

Web browser: (e.g. Chrome 88.0.4324) NA

---

📝 Description

Custom OS settings (profiles) deployed to hosts will reach a status of Verifying but never Verified. Disk encryption sits at Enforcing (pending) for longer than expected, but also never seems to get past Verifying. This is affecting both macOS and Windows hosts.

👣 Reproduction steps

1. Ensure that macOS and Windows MDM is turned ON
2. From OS settings > Disk encryption, Turn on Disk encryption
3. From OS settings > Custom settings, upload profiles for both macOS and Windows hosts
4. Ensure that at least one each of macOS and Windows hosts is enrolled
5. Observe OS settings statuses for enrolled hosts

🧑‍💻  Expected behavior

All profiles (and encryption) should reach either a Verified or Failed status.

💥  Actual behavior

More info

Note: a profile that is expected to fail will reach Failed status. Custom settings do not seem to be stuck Pending.

[roperzh]

Based on the logs provided via Slack, this is what I see:

Bitlocker

In the logs for the windows host, the Windows APIs are reporting disk encryption "in progress" (the volume is partially encrypted.) Either:

1. the encryption is taking long
2. a bug in those APIs
3. a bug in how we parse the result from those APIs
4. some weirdness/action required in the machine? maybe a restart of sorts?

Afterwards, Sabrina reported that the encryption eventually succeeded, which confirms 1.

Profile verification

In the Windows machine I don't see we ever sent the mdm_config_profiles_windows detail query. In the macOS machine, I see mdm_config_profiles_darwin ran around 2024-02-08T13:27:40-05:00, reporting the following:

{"display_name":"Disk encryption","identifier":"com.fleetdm.fleet.mdm.filevault","install_date":"2024-02-08 16:50:49 +0000"},{"display_name":"Fleetd configuration","identifier":"com.fleetdm.fleetd.config","install_date":"2024-02-07 18:31:34 +0000"},{"display_name":"Disc Recording Test","identifier":"com.github.erikberglund.ProfileCreator.16096540-2B68-4E53-89A0-C2C66621A3E0","install_date":"2024-02-08 16:51:19 +0000"},{"display_name":"Energy Saver Test","identifier":"com.github.erikberglund.ProfileCreator.A53326A0-E9D3-49A5-96F2-ADF09ACB22E5","install_date":"2024-02-08 16:51:18 +0000"},{"display_name":"MbriM enrollment","identifier":"com.fleetdm.fleet.mdm.apple","install_date":"2024-01-25 14:39:50 +0000"}

[roperzh]

I can reproduce this bug in fleet-v4.44.0, changing the tag to ~released-bug. The root of the problem is the SQL query we modified to verify profiles, I can see this error in the server logs:

SELECT list is not in GROUP BY clause and contains nonaggregated column 'fleet.mwcp.syncml' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by"

[sabrinabuckets]

Verified issue resolved. Profiles able to reach Verified and Failed statuses.

[fleet-release]

Settings floating, lost, In clouds, find verified peace. Security increased.