ADE hosts might not get the bootstrap package if they're deleted from the UI and turn on MDM features afterwards

[roperzh]

Fleet version: 4.46.1

---

💥  Actual behavior

If an ADE host is deleted from the UI, and has been assigned to Fleet for a long time in ABM, it won't get the bootstrap package delivered the next time the host enrolls.

🧑‍💻  Steps to reproduce

1. Turn on MDM features for a host using ADE
2. Observe how we send a command to install the bootstrap package
3. Delete the host from the UI
4. Turn on MDM features again
5. Observe how we don't send a command to install the bootstrap package

🕯️ More info (optional)

1. Unless we hit this clause, a cursor is used to paginate the results we get from the ABM API
2. When a device is repurposed we ask people to delete it from Fleet as well, based on this, we don't delete the entry from the table
3. Since the table is keyed by host.id , we have an orphan row
4. When the host enrolls again, we think it's not ADE-assigned, and as such we don't do any of the ADE-specific flows

[mna]

Investigating this a bit before sprint kickoff (not assigning myself just yet as I don't have the means to test those mac DEP enrollment flows by myself):

• Enrollment goes through TokenUpdate and this call returns the information whether the host is DEPAssignedToFleet and/or InstalledFromDEP: https://github.com/fleetdm/fleet/blob/dbe8f956925f111122736a415a054b5c2c4c94f0/server/service/apple_mdm.go#L2496
• Additionally, this gets the token nano enrollment information based on the device ID (which is the UUID IIRC, so the same on re-enrollment): https://github.com/fleetdm/fleet/blob/dbe8f956925f111122736a415a054b5c2c4c94f0/server/service/apple_mdm.go#L2507
• IFF the nano enrollment indicates enabled, type="device" and token_update_tally=1, we consider the request a new enrollment: https://github.com/fleetdm/fleet/blob/dbe8f956925f111122736a415a054b5c2c4c94f0/server/service/apple_mdm.go#L2512-L2513
  • This condition should always be met on re-enrollment because the nano_enrollments table is reset when a device authenticates to Fleet MDM on re-enrollment: https://github.com/fleetdm/fleet/blob/dbe8f956925f111122736a415a054b5c2c4c94f0/server/datastore/mysql/apple_mdm.go#L3333-L3338
• In addition to the previous condition, IFF the MDM check-in info indicates installed_from_dep (which comes from the host_mdm table, deleted when the corresponding hosts entry is deleted) OR dep_assigned_to_fleet (which comes from the host_dep_assignments table, which may get soft-deleted by the DEP syncer if it receives that deleted serial but it may be only when removed from ABM?) do we enqueue the post-DEP-enrollment flow: https://github.com/fleetdm/fleet/blob/dbe8f956925f111122736a415a054b5c2c4c94f0/server/service/apple_mdm.go#L2526-L2528
• That post-DEP-enrollment flow worker job then sends the command to install the bootstrap package: https://github.com/fleetdm/fleet/blob/dbe8f956925f111122736a415a054b5c2c4c94f0/server/worker/apple_mdm.go#L102

So the issue seems to be that the DEP syncer only moves forward, doesn't re-consider old devices once ingested once unless the setup assistant changed, but if we always did a full sync, it might introduce performance issues or rate limits? We already keep some DEP information in host_dep_assignments, but we only link the host via its host.id which isn't constant. Could we store the host UUID/serial and try to match using this on host_dep_assignments, and ignore it if the row has been soft-deleted (which I believe would indicate that the host is no longer assigned to Fleet in ABM)?

[fleet-release]

ADE host reborn, Fleet's embrace ensures no scorn, Tools restored, no mourn.