Baseline: 2+ major versions isn't enough

[dfabulich]

Baseline sets a target of 2+ major versions. Today, that means that the <dialog> element, released in Safari 15.4, is "baseline," because the latest version of Safari is 16.4.

I tried launching a feature with <dialog> a couple of months ago in March 2023, without a polyfill, and we got a bunch of customer support tickets from users of iOS Safari 14.

Furthermore, under the 2+ major version policy, assuming Safari 17 releases on schedule in Fall 2023, on the day Safari 17 drops, every feature from Safari 16 will be officially "baseline." For the rest of that month, no more than 50% of users will actually have Safari 17 yet; the "2+ major versions" policy will effectively drop down to 1 major version.

"Baseline" should be 3+ or 4+ major versions.

[dfabulich]

@foolip I'm not 100% sure what you're asking here. I've filed this as https://github.com/dfabulich/baseline-calculator/issues/3 and maybe we can follow up there.

[dfabulich]

Good news! I've published a year-by-year cohort analysis on https://github.com/dfabulich/baseline-calculator, and it does show improvement over time.

I'm now convinced that we can define Baseline as "release + 30 months" rather than "release + 48" months.

Focusing just on the "80% of features" column, here are the results for all cohort years:

Year	80% share	90% share	95% share	97% share	98% share	99% share
2015	19 months	42 months	54 months	83 months	never	never
2016	19 months	36 months	48 months	never	never	never
2017	10 months	24 months	44 months	never	never	never
2018	4 months	16 months	36 months	never	never	never
2019	2 months	14 months	40 months	never	never	never
2020	0 months	8 months	30 months	never	never	never
2021	2 months	10 months	never	never	never	never
2022	2 months	never	never	never	never	never
2023	never	never	never	never	never	never

Based on this, it seems plausible that when defining Baseline by a number N months, we should consider only the latest cohort year for determining "N".

80% of 2015's Baselineable features took 54 months to reach 95% market share. 80% of 2020's Baselineable features took just 30 months to get there.

Based on this data, we should define Baseline based on the latest cohort year in which 80% of features have achieved 95% market share. Currently, that's 30 months for 2020, but we can update this in future years based on new data. (We don't yet have 30 months of data for features released in 2021.)

[tidoust]

Closing this issue as generally addressed: The group rolled back on the "2+ major versions" approach and adopted a new baseline definition, thanks to the research conducted here (and elsewhere). Additional issues may be created to track more specific points.