Custom OS settings: "include any label" option for custom target

noahtalerman commented 2 months ago

Goal

User story
As and IT admin,
I want to target configuration profiles to hosts using a custom "include any label" option
so that I can have more control over which hosts, within the same configuration baseline (aka team), get certain OS settings applied.

Objective

None. Not tied to a quarterly objective.

@noahtalerman: Why are we prioritizing it? Because some workflows are blocked for a Fleet customer (see original request)

Original request

22028

Context

Product designer: @marko-lisica

Changes

Product

[ ] Reference documentation changes: Docs changes covered by "YAML changes" and "REST API changes" below.
[ ] UI changes: Figma link
[ ] CLI (fleetctl) usage changes: Update fleetctl gitops to support include_labels_any option. Specified in YAML changes.
[ ] YAML changes: #23648
[ ] REST API changes: #23647
[ ] Fleet's agent (fleetd) changes: No fleetd changes.
[ ] Activity changes: No activity changes.
[ ] Permissions changes: No permission changes. Already covered in permissions guide with "Create, edit, resend and delete configuration profiles for macOS and Windows hosts" row.
[ ] Changes to paid features or tiers: Only available in Fleet Premium. Only Fleet Premium users can add custom targets to profiles: labels_include_all, labels_include_any or labels_exclude_any.
[ ] Once shipped, requester has been notified

Engineering

[ ] Feature guide changes: TODO
[ ] Database schema migrations: TODO
[ ] Load testing: TODO

ℹ️ Please read this issue carefully and understand it. Pay special attention to UI wireframes, especially "dev notes".

QA

Risk assessment

Requires load testing: NO
Risk level: Low

Manual testing steps

Create several Labels, both Dynamic and Manual
Assign them to hosts, making sure to test Win and macOS
Upload Config profiles (Win & macOS) and assign labels, using the new Include Any target option
Ensure config profiles get applied accordingly
Test existing target options include all and exclude any to ensure no regression

Confirm the following:

[x] Fleet UI Design and Copy match Figma
[x] GitOps workflow
[x] API

Testing notes

Confirmation

[ ] Engineer (@____): Added comment to user story confirming successful completion of QA.
[x] QA (@PezHub & @jmwatts): Added comment to user story confirming successful completion of QA.

noahtalerman commented 2 months ago

Hey @zayhanlon heads up that I filed this user story for the associated customer request here: #22028

@marko-lisica can you please take this user story this design sprint? I assigned you I added it to the "Ready" column in order or priority.

zayhanlon commented 2 months ago

@marko-lisica @noahtalerman https://us-65885.app.gong.io/call?id=995886584758637603&highlights=%5B%7B%22type%22%3A%22SHARE%22%2C%22from%22%3A409%2C%22to%22%3A648%7D%5D

noahtalerman commented 2 months ago

[ ] YAML changes: #22260

Hey @marko-lisica, when you get the chance can you please close this PR and open a new one against the reference docs branch?

Also, if this PR (and the API design PR) is ready for review please mark it ready for review (not draft). So that @rachaelshaw knows it's ready for a review.

We no longer need to keep these in draft. I think we used to mark these PRs as drafts to avoid inflating the PR open time KPI. Now that these PRs are to the reference docs branch, we can merge them before the release. This way we avoid inflating the PR open time.

noahtalerman commented 2 months ago

@marko-lisica I made a couple minor tweaks to the copy in the UI (check out Loom here if you want to see)

I think the UI changes are ready to go. I assigned this one back to you. Please feel free to move this one to ready for specs after you get to the above house keeping items.

georgekarrv commented 1 month ago

Hey team! Please add your planning poker estimate with Zenhub @dantecatalfamo @ghernandez345 @gillespi314 @jahzielv @mna @roperzh

noahtalerman commented 1 month ago

Hey @zayhanlon heads up, this user story didn't make it into the upcoming engineering sprint due to capacity.

It's still prioritized. We left it on the drafting board so that it can be pulled into the next engineering sprint.

marko-lisica commented 3 weeks ago

Hey @gillespi314, heads up, I changed previus API design PR with new one. Same changes, but new one is branched off of docs-v4.60.0 branch.

marko-lisica commented 3 weeks ago

@gillespi314 Same thing for YAML changes PR. Replaced it with a new one that's branched off of docs-v4.60.0, but it includes exactly the same changes.

jmwatts commented 1 week ago

QA Notes:

Tested with “No team”: [PASS] Include All - If there’s one Label in the list that no host in “No team” should have - no profiles should be installed

[PASS]Include All - If all of the selected labels in the list would be present on a host in “No team” - profiles should be installed on that host, but not others that don’t have all labels

[PASS]Include Any - If all hosts in “No team” would have at least one of the labels - profiles should be installed on all hosts in “No team”

[PASS]Include Any - If only some hosts in “No team” would have at least one of the labels - only hosts in “No team” with matching labels should have profiles installed

[PASS]Include Any - If hosts in “No team” would not have any of these labels - no profile should be installed

[PASS]Exclude Any - If hosts in “No team” would have any of these labels - no profiles would be installed

[PASS]Exclude Any - if hosts in “No team” would not have any of these labels - profiles should be installed

[PASS]Profile previously installed but then host transferred to new team - profile should be removed

[PASS]Host transferred to “No team” and is within target label scope - profile should be installed

Tested with “Red” team: [PASS]Include All - If there’s one Label in the list that no one in “Red” team should have - no profiles should be installed

[PASS]Include All - If all of the selected labels in the list would be present on a host in “Red” team - profiles should be installed on that host, but not others in “Red” team or any other team that don’t have all labels

[PASS]Include Any - If all hosts in “Red” team would have at least one of the labels - profiles should be installed on all hosts in “Red team” but not in other teams

[PASS]Include Any - If only some hosts in “Red” team would have at least one of the labels - only hosts in “Red team” with matching labels should have profiles installed

[PASS]Include Any - If hosts in “Red” team would not have any of these labels - no profile should be installed

[PASS]Exclude Any - If hosts in “Red” team would have any of these labels - no profiles would be installed

[PASS]Exclude Any - if hosts in “Red” team would not have any of these labels - profiles should be installed on all hosts in “Red” team

[PASS]Profile previously installed but then host transferred to a new team or “No team”- profile should be removed

[PASS]Host transferred to “Red” team and is within target label scope - profile should be installed

jmwatts commented 1 week ago

@marko-lisica the Figma for "exclude any" has: don't have any

and in the product it has: don't have any

From a user perspective this makes sense that you would want "don't" included in the bolded copy, but it was different than the Figma so I just wanted to double check. Everything else matches up.

georgekarrv commented 1 week ago

@jmwatts I would agree.

PezHub commented 1 week ago

GitOps QA test results:

Successfully added profiles to a team with existing and new "include any" custom label
Confirmed profiles installed or were excluded as expected based on label assignments

Example yaml file -

name: Labels QA Team
team_settings:
  secrets:
   - secret: “ABC123”
  features:
    enable_host_users: true
    enable_software_inventory: true
  host_expiry_settings:
    host_expiry_enabled: true
    host_expiry_window: 30
agent_options:
controls:
  macos_settings:
    custom_settings:
      - path: ../custom-config-profiles/Pez-WiFi.mobileconfig
        labels_exclude_any:
          - QA
      - path: ../custom-config-profiles/macos-date-time.mobileconfig    
        labels_include_all:
          - "QA 2"
      - path: ../custom-config-profiles/macos-firewall.mobileconfig
        labels_include_any:
          - QA
          - "Dynamic - all hosts"
policies:
queries:
software:

CLI output -

fleetctl gitops -f ~/fleetdm/gitops_configs/teamLabels.yaml                    04:45:51 PM
[+] applying MDM profiles for team Labels QA Team
[+] applying 0 software packages for team Labels QA Team
[+] applying 0 app store apps for team Labels QA Team
[+] applied 1 teams
[!] gitops succeeded

Confirmed I get the expected error when trying to apply more than one custom label type to a single profile

fleetctl gitops -f ~/fleetdm/gitops_configs/teamLabels.yaml                    04:43:30 PM
Error: applying teams: POST /api/latest/fleet/spec/teams received status 422 Validation Failed: Couldn't edit macos_settings.custom_settings. For each profile, only one of "labels_exclude_any", "labels_include_all", "labels_include_any" or "labels" can be included.

marko-lisica commented 1 week ago

From a user perspective this makes sense that you would want "don't" included in the bolded copy, but it was different than the Figma so I just wanted to double check. Everything else matches up.

Good catch @jmwatts. I just updated Figma "don't" should be bold as well.

jmwatts commented 6 days ago

API testing QA Notes: No team [PASS]POST Configuration profile, No Team, Include any - Profile is created, labels are added as "include any"

[PASS]POST Configuration profile, No Team, Include all - Profile is created, labels are added as "include all"

[PASS]POST Configuration profile, No Team, Exclude any - Profile is created, labels are added as "exclude any"

[PASS]DELETE Configuration profile (uses profile_uuid, not team specific)

with team [PASS]POST Configuration profile, with a Team, Include any - Profile is created in team, labels are added as "include any"

[PASS]POST Configuration profile, with a Team, Include all - Profile is created in team, labels are added as "include all"

[PASS]POST Configuration profile, with a Team, Exclude any - Profile is created in team, labels are added as "exclude any"

[PASS]POST Configuration profile to multiple teams (in separate requests) - profile can be created on multiple teams, different profile_uuid for each profile

[PASS]POST same configuration profile to same team multiple times (should not be able to) { "message": "Validation Failed", "errors": [ { "name": "profile", "reason": "Couldn't upload. A configuration profile with this identifier (PayloadIdentifier) already exists." } ] }

Additional tests [PASS]GET Custom OS settings (requires team_id otherwise only "No team" profiles will be returned)- all profiles are listed with correct "include any", "include all", "exclude any" as well as team if applicable

[PASS]Attempt to POST with more than one "Include all", "Include any" or "Exclude any" label settings { "message": "Bad request", "errors": [ { "name": "base", "reason": "Only one of \"labels_exclude_any\", \"labels_include_all\", \"labels_include_any\", or \"labels\" can be included." } ], "uuid": "77bdd2c5-8b53-493f-9ff1-5a2d11815f81" }

fleetdm / fleet