[APM] 500 while fetching linked span without span.subtype

[1pkg]

Describe the bug:

In the thread a potential issue with :kbn/internal/apm/traces/{traceId}/span_links/{spanId}/children endpoint was unraveled. In which while fetching linked spans APM UI will assume that span.subtype is a required field and fail when the field is not set, see the required fields definition in the code.

Steps to reproduce:

1. Run provided go script to ingest traces without subtype.
2. In Kibana go to Applications > Service Inventory > Transactions > Span details > Span Links
3. Observe 500 error

Expected behavior:

Either span.subtype shouldn't be a required field or this error should be handled gracefully by filtering out spans which don't have the field.

Screenshots (if relevant):

Provide logs and/or server output (if relevant):

GET :kbn/internal/apm/traces/{traceId}/span_links/{spanId}/children

{
    "statusCode": 500,
    "error": "Internal Server Error",
    "message": "Missing required fields span.subtype in event",
    "attributes": {
        "data": {},
        "_inspect": []
    }
}

Any additional context:

An example go script to ingest the data.

package main

import "go.elastic.co/apm/v2"

func main() {
    tracer, _ := apm.NewTracerOptions(apm.TracerOptions{})
    tr := tracer.StartTransaction("trx", "type")
    span := tr.StartSpan("span", "custom.sub", nil)
    span.End()
    for range 10 {
        // Note there is no subtype here.
        s := tr.StartSpan("span", "custom", nil)
        s.AddLink(apm.SpanLink{Trace: span.TraceContext().Trace, Span: span.TraceContext().Span})
        s.End()
    }
    tr.End()
    tracer.Flush(nil)
}

[elasticmachine]

Pinging @elastic/apm-ui (Team:APM)