open-telemetry / opentelemetry-collector

OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
4.48k stars 1.47k forks source link

Add ability to compare pcommon.Map instances for equality #11277

Open tigrannajaryan opened 1 month ago

tigrannajaryan commented 1 month ago

Is your feature request related to a problem? Please describe.

I need to write a test that verifies if 2 pmetric.Metrics instances are logically equal (deep equality), including any contained pcommon.Map instances.

The complication is that the exact same logical data may be represented in multiple ways in pdata/OTLP. For example you can have one Resource and one ResourceSlice containing several Scope/Metric. It is also possible to represent the exact same data using multiple ResourceSlices each with exact same Resource value.

To be able to perform logical comparison one approach is to first normalize pmetric.Metrics instances - combine and shuffle the data around to a single canonical representation, and then traverse the 2 normalized instances and compare side-by-side.

One of the steps in such normalization is ensuring all attribute maps have the canonical representation that can be easily compared for equality. One such canonical representation is a sorted order of attributes by key.

Note: we already have Slice.Sort for testing use cases. We need some kind of equivalent for Map.

Describe the solution you'd like

There probably a few possible solutions, one of which is the following:

Introduce Map.Sort() to sort the attributes in place and Map.AsSlice() to return a new type KeyValueSlice that efficiently points to the same underlying storage. 2 slices can then be iterated in parallel and compared.

A possible undesirable effect of introducing of such API is that it leaks the fact that Map is represented as a slice. I am not sure this is a problem though, since OTLP defines attributes as a slice, so hiding that fact is probably unnecessary.

Describe alternatives you've considered

I currently use Map.Range(), copy all elements into a slice of structs {key,value}, then sort in place, then convert the sorted slice back to Map. This is complicated and inefficient.

I am curious what the maintainers think a better approach would be. Perhaps there is an alternate I can use.

bogdandrutu commented 1 month ago

What if we generate an func Equal(other) bool for all the types?

tigrannajaryan commented 1 month ago

What if we generate an func Equal(other) bool for all the types?

That would work, provided that it is deep logical equality, i.e. for Maps it should not matter in what order the attributes are listed (presumably that is what OTLP semantics are).

It may not be as fast as the Sort() option since Equal() may need to be called repeatedly in the test and I assume it will have to sort the Map every time. I work with pretty large data sets so the difference in test speeds may be tangible.

Still, I think Equal() would be useful for tests.