Error loading an opentracing plugin

[jchanam]

;TL;DR

The plugins are only read if skipper is launched from the plugins folder.

Issue explanation

Hi!

I'm trying to load the jaeger plugin to use opentracing.

I've created an image both with skipper apps (skipper/eskip) and the plugins (.so files) on /plugins folder.

Having done this, when I run skipper it gives me some errors while trying to read the plugin file (tracing_jaeger.so).

Running from any path:

#> skipper --plugindir=/plugins --opentracing "tracing_jaeger sampler-type=const"
[APP]INFO[0000] found plugin tracing_basic at /plugins/tracing_basic.so 
[APP]INFO[0000] found plugin tracing_instana at /plugins/tracing_instana.so 
[APP]INFO[0000] found plugin tracing_jaeger at /plugins/tracing_jaeger.so 
[APP]INFO[0000] found plugin tracing_lightstep at /plugins/tracing_lightstep.so 
[APP]INFO[0000] attempting to load plugin from /plugins/tracing_instana.so 
[APP]INFO[0000] attempting to load plugin from /plugins/tracing_jaeger.so 
[APP]INFO[0000] attempting to load plugin from /plugins/tracing_lightstep.so 
[APP]INFO[0000] attempting to load plugin from /plugins/tracing_basic.so 
[APP]WARN[0000] no route source specified                    
[APP]INFO[0000] Expose metrics in codahale format            
[APP]INFO[0000] support listener on :9911                    
[APP]FATA[0000] open module tracing_jaeger.so: plugin.Open("tracing_jaeger.so"): realpath failed 

Running from /plugins path (the same behaviour is given with or without --plugindir option set):

#> skipper --opentracing "tracing_jaeger sampler-type=const"
[APP]WARN[0000] no route source specified                    
[APP]INFO[0000] Expose metrics in codahale format            
[APP]INFO[0000] support listener on :9911                    
[APP]FATA[0000] module tracing_jaeger's InitTracer function has wrong signature

#> skipper --plugindir=/plugins --opentracing "tracing_jaeger sampler-type=const"
[APP]INFO[0000] found plugin tracing_basic at /plugins/tracing_basic.so 
[APP]INFO[0000] found plugin tracing_instana at /plugins/tracing_instana.so 
[APP]INFO[0000] found plugin tracing_jaeger at /plugins/tracing_jaeger.so 
[APP]INFO[0000] found plugin tracing_lightstep at /plugins/tracing_lightstep.so 
[APP]INFO[0000] attempting to load plugin from /plugins/tracing_basic.so 
[APP]INFO[0000] attempting to load plugin from /plugins/tracing_instana.so 
[APP]INFO[0000] attempting to load plugin from /plugins/tracing_jaeger.so 
[APP]INFO[0000] attempting to load plugin from /plugins/tracing_lightstep.so 
[APP]WARN[0000] no route source specified                    
[APP]INFO[0000] Expose metrics in codahale format            
[APP]INFO[0000] support listener on :9911                    
[APP]FATA[0000] module tracing_jaeger's InitTracer function has wrong signature 

I think this has something to be with the management of the plugindir management.

I've seen that a PluginDirs (type []string) is used: https://github.com/zalando/skipper/blob/7102c46b0d1c3557968d5ac1608f2702c331dc9b/skipper.go#L373 And that it has a default value ./plugins https://github.com/zalando/skipper/blob/7102c46b0d1c3557968d5ac1608f2702c331dc9b/cmd/skipper/main.go#L433 https://github.com/zalando/skipper/blob/7102c46b0d1c3557968d5ac1608f2702c331dc9b/skipper.go#L44 and that also it is filled with the value given if the flag --pluginDir is used: https://github.com/zalando/skipper/blob/7102c46b0d1c3557968d5ac1608f2702c331dc9b/skipper.go#L796

Later, that array is passed (if opentracing flag is set) to load the plugins: https://github.com/zalando/skipper/blob/7102c46b0d1c3557968d5ac1608f2702c331dc9b/tracing/tracing.go#L61-L69

As I see there, it should try every path to try to get the plugin, and in a case of success, return the plugin.

What I feel is that it's ignoring the path given and It's only able to read the plugins when skipper is run from the same directory as the plugins are.

Could you please help me with this? If there's a bug, I don't see it, but if you give me some advice, I'll be happy to fix it and send a PR.

[szuecs]

@jchanam I am not sure, but what I know is that Go plugins are not super easy to use. For example you have to build them in the same structure you also load later, because it seems to have some path dependencies. What if you just create a docker container with your plugin put into /plugins? Does it work?

@vetinari Do you have any pointer as you might know most of this.

[jchanam]

Hi @szuecs,

that's exactly what I've done. I'm doing go get for both github.com/zalando/skipper and github.com/skipper-plugins/opentracing.

Then, I go in each folder and run the following: Skipper: GOOS=linux GOARCH=amd64 CGO_ENABLED=1 go build -o built/skipper github.com/zalando/skipper/cmd/skipper Eskip: GOOS=linux GOARCH=amd64 CGO_ENABLED=1 go build -o built/eskip github.com/zalando/skipper/cmd/eskip Opentracing: GOOS=linux GOARCH=amd64 CGO_ENABLED=1 make

Then, I join all of them in a second container (to reduce size):

# Final image.
# IMPORTANT: This version should match with the version used on the golang image
FROM frolvlad/alpine-glibc:alpine-3.5_glibc-2.24
RUN apk --no-cache add ca-certificates && update-ca-certificates
RUN mkdir -p /usr/bin
COPY --from=build-stage /go/src/github.com/zalando/skipper/built/skipper /usr/bin/skipper
COPY --from=build-stage /go/src/github.com/zalando/skipper/built/eskip /usr/bin/eskip
COPY --from=build-stage /go/src/github.com/skipper-plugins/opentracing/build/*.so /plugins/

ENV PATH $PATH:/usr/bin

CMD ["/usr/bin/skipper", "-plugindir", "/plugins"]
EXPOSE 9090 9911

[szuecs]

I am not 100% sure, but when I looked into it I build https://github.com/skipper-plugins/skipper-tracing-build to document how to build it. Did you used this example to build your plugin?

[jchanam]

Hi, I've changed completely how I build the image and it seems it's working now.

It seems that I have to be very careful about updating repositories, because they don't have the exact same versions.

Thank you so much for your support!

[szuecs]

@jchanam if you have any suggestions to make the docs better....