Document standard transaction/span types

[graphaelli]

Let's move this public document to the documentation to improve discovery, introduce versioning, and improve version control.

I propose adding these to the data model documentation but open to suggestions.

[bmorelli25]

@graphaelli Is this still something we want to do?

[graphaelli]

Yes, this is still important.

[bmorelli25]

Some thoughts and questions.

Is this Google Sheet up to date? It looks like it hasn't been updated in a while.

What's the relationship between this sheet and the span_types.json spec? The spec seems to be more up to date, but doesn't mention the span.name pattern/example or span.action.

If the Google Sheet and span_types.json spec are related, it seems like we already have two (public) sources of truth for this information. Adding asciidoc documentation would only add another source.

What's important for the user to know? We could easily generate documentation from the span_types.json spec; a rough table like this, for example:

Click to expand a very bad table that was generated

| Description of span.type | `span.type` | `span.subtype` | Description | | ---- | ---- | ---- | ---- | | Spans within the application (usually not calling an external system | app | inferred | Sampling profiler inferred spans | | | app | internal | Application generic internal span for controller/handler/processing delegation | | API custom instrumentation | custom | | API custom instrumentation | | database span | db | cassandra | Cassandra | | | db | cosmosdb | Azure CosmosDB | | | db | db2 | IBM DB2 | | | db | derby | Apache Derby | | | db | dynamodb | AWS DynamoDB | | | db | elasticsearch | Ealsticsearch | | | db | graphql | GraphQL | | | db | h2 | H2 | | | db | hsqldb | HSQLDB | | | db | ingres | Ingres | | | db | mariadb | Maria DB | | | db | memcached | Memcached | | | db | mongodb | MongoDB | | | db | mssql | Microsoft SQL Server | | | db | mysql | MySQL | | | db | oracle | Oracle Database | | | db | postgresql | PostgreSQL | | | db | redis | Redis | | | db | sqlite | SQLite | | | db | unkown | Unkown database | | Request to external service, usually in request/response pattern | external | dubbo | Apache Dubbo | | | external | grpc | gRPC | | | external | http | HTTP client | | | external | ldap | LDAP client | | Messaging | messaging | azurequeue | Azure Queue | | | messaging | azureservicebus | Azure Service Bus | | | messaging | jms | Java Messaging Service | | | messaging | kafka | Apache Kafka | | | messaging | rabbitmq | RabbitMQ | | | messaging | sns | AWS Simple Notification Service | | | messaging | sqs | AWS Simple Queue Service | | External process | process | | | | | storage | azureblob | Azure Blob Storage | | | storage | azurefile | Azure Files | | | storage | azuretable | Azure Storage Table | | | storage | s3 | AWS S3 | | Template engines (no sub-type for now as really platform-specific) | template | | | | Websockets | websocket | send |

But if the span.action and span.name are important to document, does the spreadsheet need to be updated to include those for all available types/subtypes listed in the spec?

Or, am I overthinking all of this, i.e. the spec isn't important or related, and only the fields in the google sheet are important?

cc @alexanderwert

[simitt]

Great points @bmorelli25 . I believe we should get rid of the spreadsheet and instead reflect the missing parts in the spec directly and then use this as the source of truth.

[bmorelli25]

👋 Picking this up again. I'm still trying to tease out why we want to publish this spec and which use cases would bring a user to this document. Gil initially mentioned these three goals for moving the span name/type spreadsheet to our docs (emphasis mine).

Let's move this public document to the documentation to improve discovery, introduce versioning, and improve version control.

It sounds to me like the versioning and version control problems are solved by moving from the spreadsheet to the spec — which leaves improved discovery. But I reached out the @felixbarny on Slack about publishing the spec, and he mentioned that only a few agents actually test against the spec.

So, do we want to bring attention to the spec if only a few agents use it? And if so, can someone help me understand the why?

The four potential outcomes from this issue I see are:

1. Document the spec as-is, explaining that only certain agents test against it.
2. Update the spec to include fields and examples from the spreadsheet, then doc it and explain that only certain agents test against it.
3. Don't update anything. Link to the spec and/or spreadsheet from the data-model/spans page and briefly explain what they're useful for.
4. Don't doc anything.

cc @SylvainJuge who originally created the spec.

[simitt]

I like suggestion (3); with the addition to completely retire the spreadsheet and ensure everything that is still valid is in the spec. WDYT?

[SylvainJuge]

I would also be in favor of retiring the spreadsheet.

Currently the "executable spec" in JSON only covers the span type/subtype, retiring the spreadsheet completely means quite a lot:

• adding the "action" field to the spans spec (I don't know exactly how and if it's actually used in the product, so it might not be worth the effort)
• creating an equivalent spec for transactions, should at least cover transaction type
• adding a few examples for transaction and spans naming, that would only be provided for documentation as it would be quite complex to enforce those (for example link between "execute a query on Redis" and "expect the redis span to have this name" might be platform-specific and API dependant.

The JSON document currently documents some inconsistencies, for example with the Ruby agent (see https://github.com/elastic/apm-agent-ruby/pull/1183 for details). While not all agents enforce this specification, we can still point to it and use it for documentation purposes.

Also, a while ago we started thinking about name alignment, see https://github.com/elastic/apm/issues/446 for details.

On Wed, Apr 27, 2022 at 8:57 AM Silvia Mitter @.***> wrote:

I like suggestion (3); with the addition to completely retire the spreadsheet and ensure everything that is still valid is in the spec. WDYT?

— Reply to this email directly, view it on GitHub https://github.com/elastic/apm-server/issues/3569#issuecomment-1110616430, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAF2JSR4TJKRE3KAO5TE2YTVHDQO5ANCNFSM4LVSVXWA . You are receiving this because you were mentioned.Message ID: @.***>

[bmorelli25]

Thanks! So it sounds like the path forward could look something like this:

1. Minimal docs

• [x] Link to the spec and spreadsheet from the data-model/spans page and briefly explain what each is useful for. Done in https://github.com/elastic/apm-server/pull/8056.

2. Retire the spreadsheet

• [ ] Add the span.action field to the json spec (if necessary)
• [ ] Add example transaction/span names to the spec
• [ ] Create a transaction spec that covers transaction type

3. Update the docs

• [ ] Add the transaction and span specs to the documentation