newrelic / go-agent

New Relic Go Agent
Apache License 2.0
764 stars 294 forks source link

Heavily contended lock in TraceIDGenerator.generateID #767

Open ankon opened 1 year ago

ankon commented 1 year ago

https://github.com/newrelic/go-agent/blob/046b4fc116adac2b4dec4d691b61a25b80488fc2/v3/internal/trace_id_generator.go#L52-L58

In a heavily used server a lot of segments end at the same time, and from what see in the profiles this lock is a problem.

  1. Can it be avoided at all? For instance, using some magic "per-thread" generator (as much as that is possible, but possibly just having more than one rand instance might work?)
  2. If it cannot be avoided, can it be made smaller? I see that it uses defer to unlock, which means it unlocks after the conversion of the bits to the hex representation needed in the ID. Maybe it can be explicitly unlocked at least before that?

image