Separate semantic conventions from the specification.

[jsuereth]

This should enable us to move quicker and provide better fine-grained ownership of semantic conventions going forward.

[dyladan]

Given our discussion in the SIG meeting today, maybe we should consider instead of a Semantic Conventions repository, an Instrumentation repository which contains semantic conventions as well as guidelines for instrumentation authors. This would create a single place where instrumentation authors can look to make sure they're following all recommendations without bloating the spec with non-normative guidance for every cross-language instrumentation.

---

I also don't see addressed here:

1. ECS is not signal-specific but our semantic conventions are. This might be a good time to bridge this gap by making a dictionary of signal-agnostic attribute definitions which is referenced by signal-specific semantic conventions for their use.
2. Will the tooling be moved from the build-utils repository into this one? Since the tooling is semconv-specific it might make sense for it to be maintained by the semconv authors and live in the same place.

[jsuereth]

@dyladan A few responses:

ECS is not signal-specific but our semantic conventions are. This might be a good time to bridge this gap by making a dictionary of signal-agnostic attribute definitions which is referenced by signal-specific semantic conventions for their use.

This is somewhat called out in future work. I think directionally we're moving our notion of horizontal/vertical to be more use-case based (e.g. HTTP, networking, etc.) conventions vs. signal specific. HOWEVER, for initial transitiion (to preserve git history) we would move that way as a follow on, not the initial move.

Will the tooling be moved from the build-utils repository into this one? Since the tooling is semconv-specific it might make sense for it to be maintained by the semconv authors and live in the same place.

I can directly address this. It's related to a concern @Aneurysm9 raised regarding semconv exports today.

The new directory will come with automated tooling and build-tools will be updated to work with this new directory. There are some details I'd consider OUTSIDE the OTEP (but MUST be figured out as part of the transition) for how codegen should work on a language specific basis.

If you want to see that in this OTEP, then I can move it back to draft form and meet with each language SiG to figure out details for non-breaking migration of this codegen.

On instrumentation vs. semantic conventions

I just don't think we want to expand scope of semantic conventions that far right now. That' worth a discussion of its own, but well outside the scope of this PR. I'm personally against expanding the scope of semantic conventions, but I'd be ok if we have supplementary & non-normative aides for instrumentation authors if we feel it's necessary.

Additionally, if we call the repository "Instrumentation" people may actually look for implementation, when in fact this is just a specification of sorts.

[dyladan]

I can accept most of that.

On instrumentation vs. semantic conventions

I just don't think we want to expand scope of semantic conventions that far right now. That' worth a discussion of its own, but well outside the scope of this PR. I'm personally against expanding the scope of semantic conventions, but I'd be ok if we have supplementary & non-normative aides for instrumentation authors if we feel it's necessary.

I want to push back a little bit on this point. Mostly I want to ask: if not here, where? We all agreed today that there should be a repository for this type of knowledge, but nobody seems to know where it should go. The options I can see are this repo, the specification repo, the website, or a new repo.

Between this repo and the spec repo, I think this one would be a much more obvious fit. The arguments not to include that guidance here applies to an even greater degree in the spec repo.

I think putting it in a new repo is off the table for most people, but I'll document the reasons here anyway. A new repo I think further splits the community and increases the number of places where instrumentation authors need to look in order to write good effective instrumentation. Especially after this OTEP, they would need to look here, in the spec, and in whatever guidance repo.

The same argument that it divides implementors attention also applies to the website to a lesser degree, which is targeted mostly towards end-users at the moment, not instrumentation authors. I could maybe buy the argument that this is documentation and should live on the website though.

Additionally, if we call the repository "Instrumentation" people may actually look for implementation, when in fact this is just a specification of sorts.

I'm open to a different name, but calling it "Semantic Conventions" seems to exclude the idea of instrumentation guidance in the future.

[jsuereth]

I can accept most of that.

On instrumentation vs. semantic conventions

I just don't think we want to expand scope of semantic conventions that far right now. That' worth a discussion of its own, but well outside the scope of this PR. I'm personally against expanding the scope of semantic conventions, but I'd be ok if we have supplementary & non-normative aides for instrumentation authors if we feel it's necessary.

I want to push back a little bit on this point. Mostly I want to ask: if not here, where? We all agreed today that there should be a repository for this type of knowledge, but nobody seems to know where it should go. The options I can see are this repo, the specification repo, the website, or a new repo.

Between this repo and the spec repo, I think this one would be a much more obvious fit. The arguments not to include that guidance here applies to an even greater degree in the spec repo.

I think perhaps the nuance in what I'm saying didn't make it through.

• I'm against normative instructions for instrumentation in the semantic convention repository. That's outside the scope.
• I'm ok with non-normative guidelines and aides for how to instrument. In this, I think we're in alignment.

I think putting it in a new repo is off the table for most people, but I'll document the reasons here anyway. A new repo I think further splits the community and increases the number of places where instrumentation authors need to look in order to write good effective instrumentation. Especially after this OTEP, they would need to look here, in the spec, and in whatever guidance repo.

+1. I agree that would make it harder.

Additionally, if we call the repository "Instrumentation" people may actually look for implementation, when in fact this is just a specification of sorts.

I'm open to a different name, but calling it "Semantic Conventions" seems to exclude the idea of instrumentation guidance in the future.

I don't really want to bikeshed on a new name. I think Semantic Conventions fits, and it's ok to have (non-normative) guidance on instrumentation in this repository.

Where I think we need more discussion is on where different non-normative "guidelines" belong overall. For example, Metrics provides a lot of guidelines around how best to name metrics, units and dealing with fun scenarios like time-gaps, delta to cumulative conversions, etc. Which of these belongs in the specification and which belong in semantic conventions?

My rule of thumb proposal is "Guidance on how best to implement instrumentation that abides by semantic conventions" can be in non-normative locations in the semantic convention repo. All other guidelines and aides would remain in the specification. WDYT?

[Oberon00]

I'm against normative instructions for instrumentation in the semantic convention repository. That's outside the scope.

Does this mean, they should be non-normative, or they need a different place, or they are entirely out of scope for OTel?

IMHO, formally non-normative but still prescriptive would be fine.

[tigrannajaryan]

To summarize my position:

• I agree it is a good idea to move semantic convention definitions to a new repo.
• I disagree we need to reset the version numbers to v0.0.0. I think we need to continue incrementing existing version numbers. This will prevent disruptions to many other things that rely on continuity of Otel's schema family and non-overlapping version numbers.

[tigrannajaryan]

@jsuereth thanks for updating the OTEP. I no longer see in the text whether you require that version numbers are reset and we start from 0.0.0 in the new repo. Is that the case or we can continue from the next available version number 1.21.0?

[jsuereth]

@jsuereth thanks for updating the OTEP. I no longer see in the text whether you require that version numbers are reset and we start from 0.0.0 in the new repo. Is that the case or we can continue from the next available version number 1.21.0?

I'm worried about confusion w/ specification numbers. I did remove the text that requires version numbers reset, but we have not resolved WHERE to start with version numbering. Was hoping maybe we could do a quick VC / chat to discuss best path forward there.

For everyone else, I have the initial cut of automation for generating the semconv repository preserving history from the specification: https://github.com/jsuereth/otel-semconv-test

• This is generated with a script, so I can reproduce a new one in ~3min.
• This is automatically divided from the specification, so we can ensure this is the right way to fragment the two.
• There's still work to regenerate a Makefile + github automation for this repository.

However, I hope this can clarify the details of what the new repo would look like.

[carlosalberto]

FYI @yurishkuro after some more discussion, we have decided to leave the 2.0 version change discussion out of this OTEP, leaving it as an open question here and discussing its merits after the split has happened and we are ready to release there.

[carlosalberto]

Merging this as we have enough approvals and the feedback has been addressed (either by updating the content or by adding the discussion points to the open questions section, e.g. whether we should go 2.0 for the semantic conventions when this split happens).

Let's continue discussing the details once we start working on the actual changes.