elastic / apm-server

https://www.elastic.co/guide/en/apm/guide/current/index.html
Other
1.22k stars 519 forks source link

Map OpenTelemetry AWS Lambda attributes to Elastic APM concepts #5456

Open cyrille-leclerc opened 3 years ago

cyrille-leclerc commented 3 years ago

The attributes reported by OpenTelemetry for AWS Lambda services are not mapped to Elastic concepts.

There may not be "Function as a Service" concepts yet in Elastic Observability and in Elastic Common Schema.

Elastic Common Schema addition of FaaS concepts: https://github.com/elastic/ecs/issues/1516

See:

Attribute Example Value AWS Lambda Description
faas.name "checkout"
faas.id "arn:aws:lambda:eu-west-1:124526594281:function:checkout" The value of the invocation ARN for the function, which is always available through an accessor on the Lambda Context
faas.execution "e3a60aee-77cc-4407-b4a7-3df9ef263cab" The value of the AWS Request ID, which is always available through an accessor on the Lambda Context
faas.version "$LATEST"
cloud.account.id "124526594281" In some languages, this is available as an accessor on the Lambda Context. Otherwise, it can be parsed from the value of faas.id as the fifth item when splitting on :
faas. coldstart "" ⚠️ AWS Lambda doesn't implement it (see OpenTelemetry Specifications > Semantic Conventions > Instrumenting AWS Lambda)
... ... See OpenTelemetry Specifications > Semantic Conventions > Instrumenting AWS Lambda
`` ""
`` ""

Proposed mapping (according to the Elastic APM agents faas support)

OTel attribute ECS fields Comment
service.name service.name
service.version service.version
faas.name faas.name If the attribute service.name is not available, then use the value for faas.name for the service.name field as well.
faas.id faas.id
faas.version faas.version If the attribute service.version is not available, then use the value for faas.version for the service.version field as well.
faas.instance service.node.name
faas.execution faas.execution
faas.coldstart faas.coldstart
faas.trigger faas.trigger.type
N/A cloud.service.name set this field value to lambda if cloud.provider = aws and te field faas.name exists.
N/A service.framework.name set this field value to AWS Lambda if cloud.provider = aws and te field faas.name exists.

Trace Examples

image

AWS Lambda Java / OpenTelemetry / Elastic trace example ```json [ { "_index":"apm-7.13.1-transaction-000001", "_type":"_doc", "_id":"u-a7ZnoB9upy5O7rIkZz", "_version":1, "_score":null, "fields":{ "transaction.name.text":[ "/" ], "observer.name":[ "instance-0000000004" ], "user_agent.os.full":[ "Mac OS X 10.15.7" ], "user_agent.original.text":[ "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36" ], "labels.faas_name":[ "checkout" ], "url.original.text":[ "https://himkk66r78.execute-api.eu-west-1.amazonaws.com/" ], "url.scheme":[ "https" ], "service.language.name":[ "java" ], "user_agent.os.version":[ "10.15.7" ], "labels.faas_id":[ "arn:aws:lambda:eu-west-1:124526594281:function:checkout" ], "transaction.sampled":[ true ], "user_agent.os.name":[ "Mac OS X" ], "labels.cloud_account_id":[ "124526594281" ], "transaction.id":[ "99e28b11c5b310e3" ], "trace.id":[ "f2ce45d1b4e61ba101bf60363a99a3ec" ], "http.request.method":[ "GET" ], "processor.event":[ "transaction" ], "agent.name":[ "opentelemetry/java" ], "user_agent.version":[ "91.0.4472.114" ], "labels.faas_execution":[ "e3a60aee-77cc-4407-b4a7-3df9ef263cab" ], "url.full":[ "https://himkk66r78.execute-api.eu-west-1.amazonaws.com/" ], "event.outcome":[ "unknown" ], "user_agent.original":[ "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36" ], "cloud.region":[ "eu-west-1" ], "service.name":[ "checkout" ], "service.framework.name":[ "io.opentelemetry.aws-lambda-1.0" ], "labels.faas_trigger":[ "http" ], "processor.name":[ "transaction" ], "transaction.duration.us":[ 5401751 ], "labels.faas_version":[ "$LATEST" ], "observer.version_major":[ 7 ], "user_agent.name":[ "Chrome" ], "observer.hostname":[ "2a55f89e8e96" ], "transaction.type":[ "request" ], "cloud.provider":[ "aws" ], "event.ingested":[ "2021-07-02T10:19:37.970Z" ], "timestamp.us":[ 1625221168206052 ], "url.original":[ "https://himkk66r78.execute-api.eu-west-1.amazonaws.com/" ], "@timestamp":[ "2021-07-02T10:19:28.206Z" ], "url.path":[ "/" ], "ecs.version":[ "1.8.0" ], "observer.type":[ "apm-server" ], "observer.version":[ "7.13.1" ], "url.domain":[ "himkk66r78.execute-api.eu-west-1.amazonaws.com" ], "agent.version":[ "1.2.0" ], "transaction.name":[ "/" ], "user_agent.device.name":[ "Mac" ] }, "highlight":{ "trace.id":[ "@kibana-highlighted-field@f2ce45d1b4e61ba101bf60363a99a3ec@/kibana-highlighted-field@" ] }, "sort":[ 1625221168206 ] }, { "_index":"apm-7.13.1-span-000001", "_type":"_doc", "_id":"vOa7ZnoB9upy5O7rIkZz", "_version":1, "_score":null, "fields":{ "span.destination.service.type":[ "external" ], "span.name":[ "HTTP GET" ], "destination.port":[ 443 ], "observer.name":[ "instance-0000000004" ], "labels.faas_name":[ "checkout" ], "trace.id":[ "f2ce45d1b4e61ba101bf60363a99a3ec" ], "span.duration.us":[ 4168996 ], "processor.event":[ "span" ], "agent.name":[ "opentelemetry/java" ], "destination.address":[ "aiq8loptkl.execute-api.eu-west-1.amazonaws.com" ], "event.outcome":[ "success" ], "cloud.region":[ "eu-west-1" ], "service.name":[ "checkout" ], "processor.name":[ "transaction" ], "labels.faas_version":[ "$LATEST" ], "span.id":[ "1852da85f45798c9" ], "span.subtype":[ "http" ], "observer.version_major":[ 7 ], "observer.hostname":[ "2a55f89e8e96" ], "span.type":[ "external" ], "cloud.provider":[ "aws" ], "timestamp.us":[ 1625221168986624 ], "@timestamp":[ "2021-07-02T10:19:28.986Z" ], "ecs.version":[ "1.8.0" ], "observer.type":[ "apm-server" ], "observer.version":[ "7.13.1" ], "labels.net_transport":[ "ip_tcp" ], "agent.version":[ "1.2.0" ], "labels.http_flavor":[ "1.1" ], "parent.id":[ "99e28b11c5b310e3" ], "span.destination.service.name":[ "https://aiq8loptkl.execute-api.eu-west-1.amazonaws.com" ], "span.destination.service.resource":[ "aiq8loptkl.execute-api.eu-west-1.amazonaws.com:443" ] }, "highlight":{ "trace.id":[ "@kibana-highlighted-field@f2ce45d1b4e61ba101bf60363a99a3ec@/kibana-highlighted-field@" ] }, "sort":[ 1625221168986 ] }, { "_index":"apm-7.13.1-transaction-000001", "_type":"_doc", "_id":"tea7ZnoB9upy5O7rE0Ye", "_version":1, "_score":null, "fields":{ "transaction.name.text":[ "/" ], "observer.name":[ "instance-0000000004" ], "user_agent.original.text":[ "okhttp/4.9.1" ], "labels.faas_name":[ "antifraud" ], "url.original.text":[ "https://aiq8loptkl.execute-api.eu-west-1.amazonaws.com/" ], "url.scheme":[ "https" ], "service.language.name":[ "java" ], "labels.faas_id":[ "arn:aws:lambda:eu-west-1:124526594281:function:antifraud" ], "transaction.sampled":[ true ], "labels.cloud_account_id":[ "124526594281" ], "transaction.id":[ "79c9950a5c7e2d89" ], "trace.id":[ "f2ce45d1b4e61ba101bf60363a99a3ec" ], "http.request.method":[ "GET" ], "processor.event":[ "transaction" ], "agent.name":[ "opentelemetry/java" ], "user_agent.version":[ "4.9.1" ], "labels.faas_execution":[ "e0394c93-339c-422a-a144-f42895962706" ], "url.full":[ "https://aiq8loptkl.execute-api.eu-west-1.amazonaws.com/" ], "event.outcome":[ "unknown" ], "user_agent.original":[ "okhttp/4.9.1" ], "cloud.region":[ "eu-west-1" ], "service.name":[ "antifraud" ], "service.framework.name":[ "io.opentelemetry.aws-lambda-1.0" ], "labels.faas_trigger":[ "http" ], "processor.name":[ "transaction" ], "transaction.duration.us":[ 444615 ], "labels.faas_version":[ "$LATEST" ], "observer.version_major":[ 7 ], "user_agent.name":[ "okhttp" ], "observer.hostname":[ "2a55f89e8e96" ], "transaction.type":[ "request" ], "cloud.provider":[ "aws" ], "event.ingested":[ "2021-07-02T10:19:34.045Z" ], "timestamp.us":[ 1625221172517330 ], "url.original":[ "https://aiq8loptkl.execute-api.eu-west-1.amazonaws.com/" ], "@timestamp":[ "2021-07-02T10:19:32.517Z" ], "url.path":[ "/" ], "ecs.version":[ "1.8.0" ], "observer.type":[ "apm-server" ], "observer.version":[ "7.13.1" ], "url.domain":[ "aiq8loptkl.execute-api.eu-west-1.amazonaws.com" ], "agent.version":[ "1.2.0" ], "parent.id":[ "1852da85f45798c9" ], "transaction.name":[ "/" ], "user_agent.device.name":[ "Other" ] }, "highlight":{ "trace.id":[ "@kibana-highlighted-field@f2ce45d1b4e61ba101bf60363a99a3ec@/kibana-highlighted-field@" ] }, "sort":[ 1625221172517 ] }, { "_index":"apm-7.13.1-span-000001", "_type":"_doc", "_id":"tua7ZnoB9upy5O7rE0Ye", "_version":1, "_score":null, "fields":{ "span.destination.service.type":[ "external" ], "span.name":[ "HTTP GET" ], "destination.port":[ 443 ], "observer.name":[ "instance-0000000004" ], "labels.faas_name":[ "antifraud" ], "trace.id":[ "f2ce45d1b4e61ba101bf60363a99a3ec" ], "span.duration.us":[ 430418 ], "processor.event":[ "span" ], "agent.name":[ "opentelemetry/java" ], "destination.address":[ "www.google.com" ], "event.outcome":[ "success" ], "cloud.region":[ "eu-west-1" ], "service.name":[ "antifraud" ], "processor.name":[ "transaction" ], "labels.faas_version":[ "$LATEST" ], "span.id":[ "c1e74c19a8a0efad" ], "span.subtype":[ "http" ], "observer.version_major":[ 7 ], "observer.hostname":[ "2a55f89e8e96" ], "span.type":[ "external" ], "cloud.provider":[ "aws" ], "timestamp.us":[ 1625221172519411 ], "@timestamp":[ "2021-07-02T10:19:32.519Z" ], "ecs.version":[ "1.8.0" ], "observer.type":[ "apm-server" ], "observer.version":[ "7.13.1" ], "labels.net_transport":[ "ip_tcp" ], "agent.version":[ "1.2.0" ], "labels.http_flavor":[ "1.1" ], "parent.id":[ "79c9950a5c7e2d89" ], "span.destination.service.name":[ "https://www.google.com" ], "span.destination.service.resource":[ "www.google.com:443" ] }, "highlight":{ "trace.id":[ "@kibana-highlighted-field@f2ce45d1b4e61ba101bf60363a99a3ec@/kibana-highlighted-field@" ] }, "sort":[ 1625221172519 ] } ] ```
gbamparop commented 2 years ago

cloud.service.name should also be added as the UI checks for it (for a value of lambda) in order to display the lambda service metadata icon.