search

open-telemetry / opentelemetry-java-instrumentation

OpenTelemetry auto-instrumentation and instrumentation libraries for Java
https://opentelemetry.io
Apache License 2.0
1.94k stars 851 forks source link

Automatically generate AWS SDK field / attribute mappings #2291

Open ghost opened 3 years ago

ghost commented 3 years ago

Is your feature request related to a problem? Please describe. Currently AWS SDK field to span attribute mappings are hard coded for DynamoDB. It would be good to have those generated automatically from a YAML file for all AWS SDK request types.

Describe the solution you'd like Spec: https://github.com/open-telemetry/opentelemetry-specification/pull/1422/files Existing prop generator: https://github.com/open-telemetry/opentelemetry-java/blob/main/buildscripts/semantic-convention/generate.sh

Describe alternatives you've considered

ghost commented 3 years ago

@anuraaga FYI :)

eric-spence-code commented 2 years ago

@kubawach, I would be interested in working on this one.

For the auto-generation portion of this, I am thinking of targeting the following types of files. For example:

v1: https://github.com/aws/aws-sdk-java/blob/a73f09cf6f74b5a3385f2ce2622f63bd26f2e80f/aws-java-sdk-models/src/main/resources/models/sqs-2012-11-05-model.json

v2: https://github.com/aws/aws-sdk-java-v2/blob/ee3d58396f580ab5393612f65c9ef55cb08d56cc/services/sqs/src/main/resources/codegen-resources/service-2.json

I am interested if you have any thoughts on not instrumenting specific fields like the message in an SQS SendMessage or the Dynamodb item in a PutItem request? I would assume some deny list in the code-gen file, but curious if you had any thoughts on it or prior art you were thinking about following. Also curious about your opinions on adding hints for the code generator on messaging spec on SQS or DB spec on Dynamodb or aurora?

pingleig commented 1 month ago

For context of anyone finding this issue, the hard coded parameters are

https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/1c1ca91643f155440c53b6a372f826a2a05e95b5/instrumentation/aws-sdk/aws-sdk-2.2/library/src/main/java/io/opentelemetry/instrumentation/awssdk/v2_2/AwsSdkRequest.java#L22-L122

In X-Ray's Java SDK, AWS SDK attributes extractor are also hard coded but using JSON files such as https://github.com/aws/aws-xray-sdk-java/blob/8161976ab0495a1b3d9a1d87a0bbe08fcc75dcc9/aws-xray-recorder-sdk-aws-sdk/src/main/resources/com/amazonaws/xray/handlers/DefaultOperationParameterWhitelist.json#L3 and used in https://github.com/aws/aws-xray-sdk-java/blob/3c15d4030e0381bef9445c2fa2ce78aad184d10e/aws-xray-recorder-sdk-aws-sdk-v2/src/main/java/com/amazonaws/xray/interceptors/TracingInterceptor.java#L143