Offer 0-day support for new Apple operating systems

fleetdm / fleet

Open-source platform for IT, security, and infrastructure teams. (Linux, macOS, Chrome, Windows, cloud, data center)

https://fleetdm.com

Other

3.12k stars 431 forks source link

Offer 0-day support for new Apple operating systems #20086

Closed lukeheath closed 2 months ago

lukeheath commented 4 months ago

Problem

When Apple releases new versions of its operating system, it sometimes causes bugs in write operations on the host like scripts and configuration profiles.

Jamf offerts same-day support.

Potential solutions

Determine how/when to get a pre-release of a new Apple operating system.
Define ritual to ensure this continues to happen whenever a new OS update is released.
Conduct smoke testing on the new OS version and report/fix any bugs as priority items.
Confirm passing smoke tests with no new OS version-specific bugs before the OS version is pushed out to general release, and announce 0-day support.

noahtalerman commented 4 months ago

Hey @lukeheath, this makes sense to me. Do we have a QA DRI? I think it makes sense for that DRI to own this: gather instructions, define the ritual, etc.

What do you think? Moving this request to feature fest for now.

Leaving a couple questions for whoever ends up owning this here:

Do we do this ritual for Apple's minor/patch releases? Maybe we start with major? I'm assuming this request is inspired by the upcoming macOS Sequoia (15) release.
Do we have smoke tests defined for iOS and iPadOS?

lukeheath commented 4 months ago

@PezHub suggested this and has already arranged some testing, so I suggest he be the DRI for this.

(cc @georgekarrv)

PezHub commented 4 months ago

@noahtalerman for now, this would just be for major releases, like the upcoming macOS Sequoia you mentioned.
I'm not sure if @xpkoala has added smoke tests for iOS and iPadOS yet but that's a good call out since we'll want to do the same for those major releases

noahtalerman commented 4 months ago

Makes sense. I added this issue to the product board and added the story label.

I think we can take it through drafting => estimation like other stories.

@PezHub want to take a crack at updating this issue to use the user story issue template?

Once the issue is updated and the plan is filled out I think it makes sense to bring this one to an upcoming MDM design review (drafting process).

@georgekarrv and I can give feedback live that call.

noahtalerman commented 3 months ago

George: To do 0 day support we would do something like the following:

When the release candidate is out, we prioritize firing up macOS 15 and fixing bugs.
Con here is we can't address as many roadmap items

George: Alternative (3-6 week support) is we wait until macOS 15 is out and prioritize fixing bugs.

Con is we don't catch bugs before customers do

@lukeheath and @zayhanlon what do you think?

lukeheath commented 2 months ago

@noahtalerman @zayhanlon I'm unclear how important this is to our customers. Of course we ideally want to be fully tested for 0 day support, but if that means bumping requested features to the next sprint it may not be worth the trade off. Right now, I think the disruption to our roadmap would not be worth it unless there is strong demand for 0 day support.

zayhanlon commented 2 months ago

i'm in agreement @lukeheath

noahtalerman commented 2 months ago

I think the disruption to our roadmap would not be worth it unless there is strong demand for 0 day support.

@zayhanlon and @lukeheath agreed.

Maybe the following can be our plan?

When Apple ships macOS 15, the QA DRI for the MDM product group (@PezHub) opens a smoke testing ticket to test Fleet against macOS 15.
Smoke tests are completed as part of whatever the current engineering sprint is and bugs are filed and we prioritize according (using P- labels as needed).

What do y'all think?

cc @georgekarrv

georgekarrv commented 2 months ago

Yes that's what the current process is (which equates to day 7-15 support and not 0) that I am totally fine with

lukeheath commented 2 months ago

Sounds like a plan! I am closing this issue.

fleet-release commented 2 months ago

Apple's OS updates, Fleet keeps pace, secure and swift, In a cloud city's drift.

nonpunctual commented 1 month ago

@lukeheath @noahtalerman @georgekarrv @zayhanlon @PezHub

Totally understand this decision.
Within 14d is reasonable as  allows up to 90d deferral for major OS updates.
Question asked above about how important this is to customers
- Not sure I have the answer to that question, but, this is important enough to some of our competitors to advertise it:

https://www.kandji.io/blog/day-1-support-apple-new-oses https://www.jamf.com/blog/what-does-same-day-support-really-mean/ https://www.jamf.com/resources/press-releases/jamf-announces-same-day-support-sept-2024/

Here is snippet of conversation on this topic with a customer to whom this is important:

lukeheath commented 1 month ago

@nonpunctual Thanks for the feedback / context. It will help inform future decisions. Based on our current Q3 priorities, we decided not to offer 0-day support for MacOS 15, but by the time MacOS 16 I expect we will be able to offer 0-day support.