Framework for defining the maturity of OpenTelemetry and its SIGs

[jpkrohling]

On 08 Mar 2023, the OpenTelemetry GC and TC held an OpenTelemetry Leadership summit, discussing various topics. One of the themes we discussed was establishing standard rules for describing the maturity of the OpenTelemetry project. This OTEP summarizes what was discussed there and is intended to have the wider community provide feedback.

This OTEP builds on what was previously communicated by the project, especially on the following page: https://opentelemetry.io/docs/reference/specification/versioning-and-stability.

The Collector’s stability levels inspired the maturity levels.

Signed-off-by: Juraci Paixão Kröhling juraci@kroehling.de

[tigrannajaryan]

Do we beed definition of tiers and maturity/stability to be in one OTEP? Could we decouple these two discussions?

[jpkrohling]

Folks, thank you for the discussion and insights so far! Based on yesterday's GC/TC call, I reworked this OTEP to remove the parts where we determine which SIGs would be part of a future graduation, leaving only the definition of the maturity levels.

I ask you to reread the OTEP and provide your input on the open questions. Thank you again!

I'll address the open comments individually later, some/most of them might not be relevant anymore given the latest changes.

[tigrannajaryan]

@jpkrohling do you plan to continue working on this? I think it is important to standardize the maturity levels.

[jpkrohling]

@tigrannajaryan, yes, I'm catching up with my notifications this week, but I should work on this again soon.

[jpkrohling]

I think we are missing the equivalent of "Feature Freeze/RC/Frozen" level that we have in some SIGs

I added a new section for this between beta and stable.

[jpkrohling]

This PR is now ready for another review.

[tigrannajaryan]

@jpkrohling please fix the build.

[jpkrohling]

please fix the build.

@tigrannajaryan, fixed, thanks for pointing that out!

[tigrannajaryan]

@open-telemetry/specs-approvers please review. I think it is important to have common maturity levels and this OTEP is an improvement over what we have.

[tigrannajaryan]

Let's keep this open for a while. I want to see significantly more approvals on this since it impacts all Otel components.

[tigrannajaryan]

This looks good to me.

Are there any next steps planned after the merge of this OTEP? Do we expect all maintainers to self-assess their deliverables?

In the past for new requirements like this I filed issues in every repo.

[tigrannajaryan]

This will impact all languages. Maintainers, please review/approve.

@open-telemetry/java-maintainers @open-telemetry/go-maintainers @open-telemetry/javascript-maintainers @open-telemetry/dotnet-maintainers @open-telemetry/cpp-maintainers @open-telemetry/python-maintainers @open-telemetry/php-maintainers @open-telemetry/ruby-maintainers @open-telemetry/swift-maintainers @open-telemetry/rust-maintainers @open-telemetry/erlang-maintainers

Also will impact Collector, please review @open-telemetry/collector-maintainers @open-telemetry/collector-contrib-maintainer

[jpkrohling]

@marcalff, I agree in principle with you, and I had wording like that in the beginning: the idea was that a component would be stable only when all its dependencies were stable. As it turns out, we have good examples of cases where this isn't necessarily true, like for the Collector Contrib (with its 100s of components) and the Java Agent, with its pluggable pieces. So, that specific wording has been removed.

One way we solved the situation you mentioned was to have a notion of "tiered" features, stating that all "tier-1" features have to be stable before a component can be called stable. New features could start life as a tier-2 feature, eventually being promoted to tier-1 upon reaching a stable level, perhaps once all dependent parts supported it. We dropped that, as it seemed too complicated and restrictive for the moment. We might revive that part at a later point.

Given that we care the most about the user-facing side of this story, the idea now is that components and SIGs can define what's their external messaging, taking the state of their dependencies only as a source of information for them to make the decision.

[austinlparker]

One thing I didn't notice in this, and really nowhere in our maturity model, is documentation. Should that be split into its own OTEP?

[jpkrohling]

I like having documentation as part of the definition of done, but I'd leave it for individual SIGs to determine. A separate OTEP with recommendations would be nice, though.

[tigrannajaryan]

@jpkrohling can you please address/resolve all comments?

Everyone who reviewed so far and haven't approved: if you have objections please speak up / request changes to block the OTEP (or approve the OTEP if you agree with it).

[jpkrohling]

Folks, I marked a few conversations as resolved, as I believe they have been sufficiently addressed or discussed. As of right now, there are only two open items, one related to components "under development" being removed without notice, and one about stability guarantees for "stable" deliverables.

If you think there's still something to address in this OTEP, please reopen a discussion or open a new one.

[jpkrohling]

Folks, what's missing to merge this one?

[tigrannajaryan]

Folks, what's missing to merge this one?

All comments must be resolved before the PR can be merged.

[jpkrohling]

@tigrannajaryan, I marked the last comments as resolved, as they are either being tracked by follow-up issues, or have been answered without further questions after at least a few days.

I think we are at a stage now where this should be merged, given we have more than 10 approvals. Further tweaks can be made via follow-up PRs.

[tigrannajaryan]

All comments resolved, enough approvals, 2 days since last comment, all merging conditions met. Merging.

[tigrannajaryan]

Thank you @jpkrohling and everyone who helped improve this OTEP!

[tigrannajaryan]

@jpkrohling I think it can be useful to announce this OTEP to all maintainers, perhaps even file issues in SDK repos so that they start labeling components according to the OTEP.

[jpkrohling]

Absolutely, I'm adding this to my to-do list, scheduled for executing in a week or so, to give time for everyone to come back from their PTOs and catch-up with the notifications.