fix: cure large spans by truncating big tag values to a reasonable size

[tim-mwangi]

Description

For the kafka exporter, large spans - by default those over 1MB - cause an error in exporting and end up in the retry queue where they hold up the memory, eventually causing the process to run out of allocated memory and crash. We'd like to be able to "cure" the large spans by truncating the tag values to 128KiB(which is configurable)

Testing

• comprehensive unit tests

Checklist:

• [ ✅ ] My changes generate no new warnings
• [✅ ] I have added tests that prove my fix is effective or that my feature works

[tim-mwangi]

@davexroth @sanjay-nagaraj it's possible to "cure" the spans while exporting if they are projected to exceed the Producer.MaxMessageBytes value.

[sanjay-nagaraj]

How expensive is this ?

[tim-mwangi]

How expensive is this ?

Only affects spans larger than 1MB and it's a loop on the tags with a simple truncation and replacement of values. For each truncated attribute we add ".truncated" tag as well.