"Device Enrollment" notification isn't showing up for macOS hosts in Apple Business Manager (ABM)

[noahtalerman]

Fleet version: Observed in Fleet's dogfood environment (commit 66b992e)

---

💥  Actual behavior

A macOS host, that's in Apple Business Manager (ABM) and assigned to dogfood, had MDM turned off: the automatic enrollment profile was removed.

When we went to the notification center, the "Device Enrollment" notification never appeared.

Noah: I think we expect the "Device Enrollment" notification to appear automatically because Fleet is running a script at some internal to trigger the notification.

This can be observed in the Gong recording here starting at 16:42.

🧑‍💻  Steps to reproduce

1. Turn off MDM for a macOS host that's assigned to MDM solution in ABM
2. Wait to see if the "Device Enrollment" notification appears

More info

This was observed on @zwass's macOS host. If you want to pull logs or troubleshoot using this Mac, please contact @zwass.

🛠️ To fix

Document that turning off MDM breaks fleetd and no code changes are needed at this time.

[noahtalerman]

I think we expect the "Device Enrollment" notification to appear automatically because Fleet is running a script at some internal to trigger the notification.

@roperzh when you get the chance, can you please sanity check me here? I think this is still the expected behavior but maybe we changed it at some point?

[roperzh]

@noahtalerman yes, I think that's expected. This is a bug

This could be another representation of #15461 , @zwass could I bother you to help us debug this?

If you prefer a hands-off approach, just making the host go online will do, because we can grab logs and run things via scripts.

Otherwise, could you start by running sudo profiles show --type enrollment? and, if possible, grab the orbit logs located at /var/log/orbit/orbit.stderr.log?

Thank you!

[zwass]

Unfortunately I'm traveling currently and will need to do this when I return (it's not my daily-use device). I've put it on my calendar for Feb 26 when I should be back in person with that device.

[roperzh]

Zach is traveling again, we put another reminder in calendar and I'll mark this issue as blocked until then so people know to hold.

[zwass]

Apologies for the long delay, but here are the requested resources: https://drive.google.com/drive/folders/1qIhplJ3_v_ZOG8P57weu34T_NV5VAZN_

(In Google Drive because there may be confidential info)

Here is also a screenshot showing the empty profiles window:

[zwass]

@roperzh if there is more needed on my end, please try to let me know ASAP as I will be traveling again (only for a few days this time) starting midday tomorrow.

[roperzh]

@zwass thank you very much, taking a look now!

[roperzh]

@zwass @noahtalerman we found an edge case we haven't considered when we planned the manual cert renewal!

1. Zach had installed the "fleet base" package that reads an enroll secret / Fleet URL from a configuration profile
2. By unenrolling Zach, we removed the configuration profile
3. Orbit is stuck in a loop trying to search these values:

2024-03-18T18:12:01-07:00 INF didn't find configuration values in system profile, trying again in 30 seconds

By design we decided to not let fleetd proceed unless we have those two values, but we might want to reconsider this? I wonder if there's an easier solution we could do.

This is somewhat of an edge case now that we do automatic cert renewals, but I wonder if there are other scenarios that motivate a fix for this?

[zwass]

IMO we need to either commit to fixing this or document a workaround.

[noahtalerman]

we found an edge case we haven't considered when we planned the manual cert renewal!

@roperzh ad @zwass nice find!

If I'm understanding correctly, because fleetd doesn't proceed w/o enroll secret and server URL, we don't pop up the notification.

Does this mean, that if I turn off MDM for a DEP host, some fleetd features won't work for this host? If so, which ones?

we decided to not let fleetd proceed unless we have those two values

@roperzh do you remember why we decided this?

[roperzh]

@noahtalerman

If I'm understanding correctly, because fleetd doesn't proceed w/o enroll secret and server URL, we don't pop up the notification.

that's right!

Does this mean, that if I turn off MDM for a DEP host, some fleetd features won't work for this host? If so, which ones?

I think, ~1 hour after you turn off MDM, probably nothing will work, no fleet desktop, no osquery, etc.

@roperzh do you remember why we decided this?

I think the vast majority of what fleetd does needs to reach the server, even if we let it proceed you won't get much out of it (eg: if you don't have an URL, osquery can't send results, orbit can't pull notifications, fleet desktop can't do the API calls, etc)

which makes me think that the real solution for this problem is to store the enroll secret in the keychain right after we receive the profile, which is tracked in https://github.com/fleetdm/fleet/issues/16118

[noahtalerman]

@roperzh thanks for the detailed response!

makes me think that the real solution for this problem is to store the enroll secret in the keychain right after we receive the profile

This makes sense to me 👍

I think for this bug, let's document the current behavior (turning off MDM breaks fleetd) + workaround (reinstall the fleetd agent)

After that I think we can close this bug.

[fleet-release]

In cloud city's glow, A notification's missing, Fleet's code remains whole.

Mac's call echoes void, Yet in Fleet's mirror, no flaw, Scripted rhythm flows.

Understanding blooms, When MDM falls silent, Fleet's truth is revealed.