open-telemetry / opentelemetry-proto

OpenTelemetry protocol (OTLP) specification and Protobuf definitions
https://opentelemetry.io/docs/specs/otlp/
Apache License 2.0
596 stars 256 forks source link

`ExportLogsServiceRequest` JSON example may be invalid #579

Open demurgos opened 2 months ago

demurgos commented 2 months ago

I am writing serialization tests for Opentelemetry Rust, and working with the examoke JSON files in this repo.

I have concerns that the logs.json example may be invalid or out of date; or I may be missing something.

The example contains an attribute using an array of strings as values:

                {
                  "key": "array.attribute",
                  "value": {
                    "arrayValue": {
                      "values": [
                        {
                          "stringValue": "many"
                        },
                        {
                          "stringValue": "values"
                        }
                      ]
                    }
                  }
                }

The JSON above uses AnyValue for the array items, but in the protobuf specs I see ArrayValue which adds a layer of indirection (notice the extra "value" layer). I would expect the following JSON serialization:

                {
                  "key": "array.attribute",
                  "value": {
                    "arrayValue": {
                      "values": [
                        {
                          "value": {
                            "stringValue": "many"
                          }
                        },
                        {
                          "value": {
                            "stringValue": "values"
                          }
                        }
                      ]
                    }
                  }
                }
tigrannajaryan commented 2 months ago

I think you are right. Can you please double check by sending the payload to the otlp receiver in the Collector?

demurgos commented 2 months ago

I wrote a test program using Rust's OpenTelemetry SDK. The raw JSON payload is attached below. It uses the the variant with the extra {"value": ...} layer. Please note that I am not 100% confident in the correctness of Rust's exporter though (I was actually testing it when I stumbled upon the issue).

JSON payload ```json { "resourceLogs": [ { "resource": { "attributes": [ { "key": "service.name", "value": { "stringValue": "eternaltwin" } }, { "key": "deployment.environment", "value": { "stringValue": "dev" } }, { "key": "telemetry.sdk.version", "value": { "stringValue": "0.24.1" } }, { "key": "telemetry.sdk.name", "value": { "stringValue": "opentelemetry" } }, { "key": "telemetry.sdk.language", "value": { "stringValue": "rust" } }, { "key": "service.version", "value": { "stringValue": "0.14.2" } } ], "droppedAttributesCount": 0 }, "scopeLogs": [ { "scope": { "name": "test_logger", "version": "", "attributes": [], "droppedAttributesCount": 0 }, "logRecords": [ { "timeUnixNano": "0", "observedTimeUnixNano": "1725057088789852647", "severityNumber": 0, "severityText": "", "body": null, "attributes": [ { "key": "test_array", "value": { "arrayValue": { "values": [ { "value": { "stringValue": "foo" } }, { "value": { "stringValue": "bar" } } ] } } } ], "droppedAttributesCount": 0, "flags": 0, "traceId": "", "spanId": "" } ], "schemaUrl": "" } ], "schemaUrl": "" } ] } ````
Test program ```rust let logger_provider = SdkLoggerProvider::builder() .with_resource({ let process = Resource::new( config .opentelemetry .attributes .value .iter() .map(|(key, value)| KeyValue::new(key.to_string(), value.value.to_string())), ); let telemetry: Resource = TelemetryResourceDetector.detect(Duration::from_secs(0)); Resource::merge(&telemetry, &process) }) .with_batch_exporter( { let exporter = opentelemetry_otlp::new_exporter() .http() .with_endpoint("http://localhost:5080/") .with_timeout(Duration::from_secs(1)) .with_http_client(HyperClient::default()) .with_headers({ let mut metadata: HashMap = HashMap::new(); metadata.insert(String::from("Authorization"), String::from("abc")); metadata }); exporter.build_log_exporter().expect("exporter is valid") }, opentelemetry_sdk::runtime::Tokio, ) .build(); let logger: SdkLogger = logger_provider.logger("test_logger"); logger.emit({ let mut r = logger.create_log_record(); r.attributes = Some(vec![( String::from("test_array").into(), AnyValue::ListAny(vec![String::from("foo").into(), String::from("bar").into()]), )]); r }); ```
tigrannajaryan commented 1 month ago

Please note that I am not 100% confident in the correctness of Rust's exporter though (I was actually testing it when I stumbled upon the issue).

This is the reason I was suggesting to send to the Collector. We consider it the canonical implementation. Can you run a Collector with otlp receiver and debug exporter and send it this payload and confirm in the debug output that values are correctly received?