Possibility to start tracing from apache http reverse proxy.

[kmalecki]

I have a situation where:

[Apache http proxy] -> [micro-service structure**(created traceId/spanId)**].

I need to start tracing requests from Apache server to bind logs from Apache to logs from micro-servces. So, when request is authorized in Apache proxy, and log is created, i need to have traceId correlated to this request,:

[Apache http proxy**(created traceId/spanId)**] -> [micro-service structure**(passed tracedId from Apache)**]

Is there any possibility to force Zipkin to trace incoming requests, starting from Apache http server proxy?

[codefromthecrypt]

I've not heard of anyone making an apache module as yet. right now, linkerd might be the only known reverse proxy that has builtin zipkin support.

[jcarres-mdsol]

We use nginx but same situation, it would be great if we could use trace_ids. Maybe this issue should be moved somewhere else thought as it is not related to the ruby client.

[jcarres-mdsol]

BTW @kmalecki maybe you would be interested on this https://github.com/JordiPolo/lorekeeper We have a middleware which takes the trace Ids and put it in here so we have nice json formatted log messages with trace_ids. It may be too much depending on how much are you building around querying your logs but it can be a life-saver.

[mjbryant]

I am also interested in an Apache module to properly start a Zipkin trace. We currently generate Apache spans quite hackily by having Apache log certain Zipkin headers and generating the spans entirely out-of-band. It is very difficult to maintain, though probably has lower in-band overhead.

[kmalecki]

I think that the only solution is to generate trace id by a module for apache and add header to request. Unfortunately there is no available module, that can generate such thing. Thank You All for tips. I will not close this issue yet, maybe I or someone else will find something else.

[adammichalik]

If it helps anyone, I achieved the above using mod_unique_id and mod_headers:

RequestHeader setifempty X-B3-TraceId "expr=%{md5:%{reqenv:UNIQUE_ID}}"
RequestHeader edit X-B3-TraceId "^(.{0,16}).*$" "$1"

%{reqenv:UNIQUE_ID} is a unique string generated by mod_unique_id. Then, it gets MD5-hashed and assigned to the X-B3-TraceId header. This gives a 32-character hex string = 128 bit traceId. Since my downstream libraries require the traceIds to be 64-bit, the RequestHeader edit extracts the first 16 characters (64 bit) from the value and reassigns it back to the X-B3-TraceId header which is sent downstream.

[codefromthecrypt]

awesome thanks for sharing @adammichalik!

[jorgheymans]

Does this warrant a small README.md blurb in https://github.com/openzipkin/brave/tree/master/instrumentation/apache-httpd or possibly https://github.com/openzipkin-contrib/apache-httpd ?

[codefromthecrypt]

maybe in b3 propagation repo, I guess, as it isn't java specific?

[sskilbred]

@kmalecki were you able to find a working solution for this? I agree with you that if you want a trace span to be generated (and not just propagate the trace context) you probably need a module to handle that.

[kmalecki]

@sskilbred No, i moved to public cloud and Datadog.

I will close this thread.

[trajano]

Can we reopen this thread? This is a useful feature to have with an Apache DMZ like https://github.com/trajano/docker-oidc-proxy

[aidofitz]

If it helps anyone, I achieved the above using mod_unique_id and mod_headers:

RequestHeader setifempty X-B3-TraceId "expr=%{md5:%{reqenv:UNIQUE_ID}}"
RequestHeader edit X-B3-TraceId "^(.{0,16}).*$" "$1"

%{reqenv:UNIQUE_ID} is a unique string generated by mod_unique_id. Then, it gets MD5-hashed and assigned to the X-B3-TraceId header. This gives a 32-character hex string = 128 bit traceId. Since my downstream libraries require the traceIds to be 64-bit, the RequestHeader edit extracts the first 16 characters (64 bit) from the value and reassigns it back to the X-B3-TraceId header which is sent downstream.

Hi, really useful thanks. Based on this, I've attempted to do a w3c traceparent header - not sure it is exact as it is intended to be but may be of use. Horribly inefficient though, should really be written as a module!

# Header injection of w3c compliant "traceparent" distributed tracing header
# https://www.w3.org/TR/trace-context/
# This is using mod_unique_id to get uuid for the trace id, and the client ip address as the span/parent id 
# Then use sha1 to convert both to 40 char hex. NB the SHA hashing is not necessary, we just want to hex 
# encode - but haven't come up with any other way to do this using Apache directives. 
# Some concerns around resource utilisation of SHA1 under high volumes, however
# according to https://automationrhapsody.com/md5-sha-1-sha-256-sha-512-speed-performance/ 
# A 2.6Ghz Core i7 can do >1million SHA1 operations/second, so this may not be major concern

RequestHeader setifempty traceparent "expr=%{sha1:%{reqenv:UNIQUE_ID}}-%{sha1:%{REMOTE_ADDR}}" 

# W3C traceparent standard is for a 32 char hex id for trace and 16 char for span, so edit
# accordingly, taking required digits from originally generated, plus add other required elements 

RequestHeader edit traceparent "^(.{32}).*-(.{16}).*$" "00-$1-$2-00"