search

cncf / prometheus-conformance

Prometheus Conformance Program
Apache License 2.0
9 stars 4 forks source link

Clarify usage of the Prometheus mark when 100% compatible #6

Open caniszczyk opened 3 years ago

caniszczyk commented 3 years ago

For example, if "All relevant compliance scores are multiplied into a compatibility score. When 100% compatibility has been reached, a versioned mark "Prometheus X.Y Compatible" will be awarded"

If this is the case, should we allow for a product to be called "Cloud Managed Prometheus" - similar to how we allow products who are certified kubernetes to use the mark in a more loose manner, i.e., "Google Kubernetes Engine"

RichiH commented 3 years ago

It's a surprisingly complex question.

As written, it would not fit: e.g. https://github.com/RichiH/modbus_exporter would pass the OM tests. As this is the only relevant test and as it's compliant, it could then call itself "Prometheus x.y compatible", which makes sense. But calling this "RichiH Prometheus" is obviously incorrect.

If we want to go there, would need to have a suite of mandatory tests, and new ones around storage, alerting, etc. We will need time to create those tests, and to build some familiarity of who gets certified, how much overhead it is, etc.

As such, I think it's too early to do it now; my long term gut still goes slightly against, but I lack data.

I will also make sure Prometheus-team is aware of this issue and chimes in.

RichiH commented 3 years ago

For reference: https://www.linuxfoundation.org/en/trademark-usage/

RichiH commented 3 years ago

When is the next GB meeting? I presume we should have at least an initial decision by that point?

caniszczyk commented 3 years ago

It will be mid July, we should iron out the details by then.

I think you need to think about balancing the incentive of products to go through this process, like Google Container Engine, morphing into Google Kubernetes Engine once certified, versus what you may want for each subcomponent. On the bright side, we have some time to figure this out.

Ideally, we'd start with 2-3 candidate conformant products/projects and everything blooms from there.

codesome commented 2 years ago

@RichiH this came up in some discussion. I was not sure myself what 100% compatible means after reading the readme. We mention that they need to pass Remote-Write tests in the readme, but most of the softwares like Thanos, Cortex, Promscale, etc, who want to be 100% Prometheus compatible, would not remote-write. So they can never be Prometheus compatible?

RichiH commented 2 years ago

Good catch. Current version is signed off by Prometheus team, but Prometheus team can also change its collective opinion. Do you want to send email for this or would you prefer if I did it?

codesome commented 2 years ago

I can send one