back-end bits for MDM Puppet module

[roperzh]

Related user story

11185

Task

We need two endpoints, one endpoint to "pre-assign" a set of profiles to a host, and a separate endpoint to match the preassigned profiles to a team, and assign the host to that team.

• [x] Premium-only endpoints

• [x] Create an API endpoint to "pre-assign" profiles to a host:

  • [x] Receives the following arguments:

  • [x] external_host_identifier (string) this is a string used by the third-party (Puppet) to reference a host. It's different from hosts.uuid

  • [x] host_uuid (string)

  • [x] profile ([]byte) raw XML containing a configuration profile. We probably need to store the raw XML in Redis (instead of storing a hash) because it's possible that a new team containing this profile needs to be created.

  • [x] group (string) (I'm not sure about the naming of this parameter) used to generate a team name

  • [x] Each time is called, it stores in Redis the values described above, keyed by external_host_identifier, mainly because that's the only value we'll get to retrieve each entry in the sibling endpoint.

  • [x] Create an API endpoint to "match" a set of pre-assigned profiles to a host:

  • [x] Receives an external_host_identifier (string)

  • [x] Fetches all the profiles that have been assigned to the host from Redis.

  • [x] Maybe (could be split up into a separate issue). Tries to match all the profiles with an existing team.

    • [x] If the team exists, assign the host to that team.
    • [x] If the team doesn't exist, create a team with the set of profiles. The team name is a combination of all the unique group values provided in the previous endpoint in sorted order, with addition of the timestamp to make the name unique (e.g. canary - workstations (2022-05-12:01:03:12)

[roperzh]

@mna worth chatting about two things:

1. Any ideas for the API endpoints? I was using POST /api/_version_/fleet/mdm/apple/profiles/preassign and POST /api/_version_/fleet/mdm/apple/profiles/match, but open to anything! just need to agree on an API.
2. The team matching is complicated enough that might grant a separate ticket. I'll ping you as soon as I'm done with the Ruby code to see if I can be of help!

[roperzh]

@mna not really helpful, but I was just getting started with this and here's what I've got so far: https://github.com/fleetdm/fleet/pull/12021

[mna]

@roperzh some comments/questions/thoughts (I should be able to start work on that first thing tomorrow morning):

• Endpoints sound good to me, is it correct to say that POST /api/_version_/fleet/mdm/apple/profiles/preassign will be called H * P times where H is the number of hosts and P the number of profiles assigned per host (although the number of Ps can vary per H but just as a loose calculation), and POST /api/_version_/fleet/mdm/apple/profiles/match is called only once per H, and is guaranteed to be called after any call to preassign?
• My initial understanding was that we'd always create the teams based on the "preassign" data received from puppet, but I think I see now that this is not a single-use puppet module that will run only when Fleet is initialized (has no existing team), right? It will possibly run more times after that, even when teams already exist in Fleet, which is why you mention looking for an existing team that has that exact set of profiles, so we don't always create new teams?
• The preassign endpoint will (at least if everything goes as planned) only access Redis, so it should be reasonably fast and able to handle the load that running puppet for multiple hosts will generate. The match endpoint will have to do quite a bit of work though. My initial understanding (again) was that we'd receive a single API request at the end, is there something that prevents that and requires that we receive one per host? I would've triggered a worker job from that single request, but that may still be a good idea to do that even if it's one request per host, as it may generate a lot of load in a small time window. At least something to keep in mind as a future improvement, and we can start by doing the work in the request handler directly.

To recap, the endpoints payload would look like that:

POST /api/_version_/fleet/mdm/apple/profiles/preassign: JSON body

{
  "external_host_identifier": "puppet-specific-unique-per-host",
  "host_uuid": "fleet-mdm-host-uuid",
  "profile": "base64-encoded-raw-xml-bytes",
  "group": "user-provided-team-name-part"
}

Just to be sure: we're certain the profile bytes payload will be properly base64-encoded as expected by Go's JSON marshaling?

POST /api/_version_/fleet/mdm/apple/profiles/match: JSON body

{
  "external_host_identifier": "puppet-specific-unique-per-host"
}

[roperzh]

@mna

Endpoints sound good to me, is it correct to say that POST /api/version/fleet/mdm/apple/profiles/preassign will be called H * P times where H is the number of hosts and P the number of profiles assigned per host (although the number of Ps can vary per H but just as a loose calculation)

Yes! absolutely correct to say that.

POST /api/version/fleet/mdm/apple/profiles/match is called only once per H, and is guaranteed to be called after any call to preassign?

It is called once per H, and we can guarantee that, when called, it will be after any call to preassign. I'm not 100% sure we can guarantee it'll always be called (ie: probably a good idea to add an expiration to the stored keys, maybe around ~30mins/1 hour?)

My initial understanding was that we'd always create the teams based on the "preassign" data received from puppet, but I think I see now that this is not a single-use puppet module that will run only when Fleet is initialized (has no existing team), right? It will possibly run more times after that, even when teams already exist in Fleet, which is why you mention looking for an existing team that has that exact set of profiles, so we don't always create new teams?

This is correct, this cycle of preassignment + matching will happen once per H every time the puppet agent checks in (we can assume ~30 minutes)

My initial understanding (again) was that we'd receive a single API request at the end, is there something that prevents that and requires that we receive one per host?

sorry, I think I wasn't clear on that! we can't do a "match" call for all the hosts because hosts check in asynchronously at different times. The mechanism is very similar to how we handle detail queries for osquery, hosts check in all the time at different times.

I would've triggered a worker job from that single request, but that may still be a good idea to do that even if it's one request per host, as it may generate a lot of load in a small time window. At least something to keep in mind as a future improvement, and we can start by doing the work in the request handler directly.

That sounds good, or to your point of having a "match" call for various jobs, we could do it in a cron job too.

[roperzh]

@mna sorry for the double ping, I see I missed two questions:

Just to be sure: we're certain the profile bytes payload will be properly base64-encoded as expected by Go's JSON marshaling?

Absolutely 👍

To recap, the endpoints payload would look like that:

both payloads look great!

[mna]

@roperzh

probably a good idea to add an expiration to the stored keys, maybe around ~30mins/1 hour?

Yes, definitely!

we can't do a "match" call for all the hosts because hosts check in asynchronously at different times. The mechanism is very similar to how we handle detail queries for osquery, hosts check in all the time at different times.

Oh interesting! My mental model of it was that a puppet run was triggered at some point and executed for all hosts, but what you describe makes a lot of sense.

Thanks for the extra details!

[fleet-release]

Puppet module blooms, Profiles matched, teams unite, Cloud city thrives, calm.