RFC : Implement new span_id API ?

[fholzhauser]

Follow up discussion from #12 There seems to be an interest in a new API which is similar to the functional API but instead of passing around the span record for log/tag/finish, it'd store the span data in some storage (e.g. ETS) and pass around the span_id instead. This would have the same (relatively high) instrumentation impact on existing code as the functional API, would be somewhat more resource consuming, but might open other possibilities e.g. for enhanced filtering to use information from all currently open spans.

[RoadRunnr]

I don't the sense in passing the SpanId in a functional fashion if it is not changing between calls to the API, why should be it be treated like the span record? IMHO the API discussed in #14 is completely sufficient for that.

I had the idea with ETS table as well. However you seem to imply that the current span record gets stored in the table. My idea is to create a new entry in the table for each log or tag call. That way most calls will only write into the ETS and only the final send will read and delete data from the table. Also, the table could be kept in a separate, possible global, management process. That way most processing would out of the normal execution path. tag and log calls would only send a message to the management process. Of course, resource cleanup would be an issue. We could use a NIF resource structure as part of the SpanId and use the resource cleanup hook as a notification function when the last reference to the SpanId is dropped.

[fholzhauser]

Indeed, we wouldn't need to pass the ID in the same variable scope. Only when calling other functions where we want to add tags/logs to the span. Storing the tags/logs as separate lines in a bag table, then collect/delete them when the span is finished makes adding tags/logs very light, but collecting the span fairly heavy. Probably still faster though than read/write a record in an ordered set for each action. I have not thought of that indeed. It sounds like a good idea if we go with ETS.

How about a different approach : spawning a process for each span and pass its PID as ID? It would make housekeeping simple to implement, probably scales well, and offloads the request handling process from e.g. filtering (should we end up with something sophisticated there). On the downside, creating that many additional processes might be a concern in some cases.

[cmullaparthi]

I like this idea, let's implement it.

[fholzhauser]

New API with individual processes per span is implemented in 0.3.0