Can't run jaeger-query behind ingress

[maxkondr]

Hi, I am trying to setup jaeger accordingly to https://github.com/jaegertracing/jaeger-kubernetes and using Ingress resource for accessing Jaeger UI. But I see some issues with UI, in particular - incorrect urls for .js|.css file requests. I tried to set query.prefix option in config for this purpose but it is ignored.

Config:

apiVersion: v1
data:
  agent: |
    collector:
      host-port: "jaeger-collector:14267"
  collector: |
    es:
      server-urls: http://elasticsearch.logging:9200
    collector:
      zipkin:
        http-port: 9411
  query: |
    es:
      server-urls: http://elasticsearch.logging:9200
    query:
      static-files: /go/jaeger-ui/
      prefix: /tracing/jaeger
  span-storage-type: elasticsearch
kind: ConfigMap
metadata:
  labels:
    app: jaeger
    jaeger-infra: configuration
  name: jaeger-configuration
  namespace: tracing

The ingress resource looks like:

apiVersion: v1
kind: List
items:
- apiVersion: extensions/v1beta1
  kind: Ingress
  metadata:
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/add-base-url: "true"
      nginx.ingress.kubernetes.io/from-to-www-redirect: "true"
      nginx.ingress.kubernetes.io/rewrite-target: /
    name: jaeger
    namespace: tracing
    resourceVersion: "542749"
  spec:
    rules:
    - host: etsys-sm-107.vms
      http:
        paths:
        - backend:
            serviceName: jaeger-query
            servicePort: jaeger-query
          path: /tracing/jaeger
    tls:
    - hosts:
      - etsys-sm-107.vms
      secretName: tracing-certs

But the urls look like:

https://etsys-sm-107.vms/static/css/main.2a0fcb74.css
https://etsys-sm-107.vms/static/js/main.b18b9217.js

I am using image: jaegertracing/jaeger-query:1.2

Please advise.

[yurishkuro]

The prefix switch only controls the API location, the UI still has to run at /

We'll be doing a new release this week that includes support for running under custom roots.

[maxkondr]

great! waiting for new one :)

[maxkondr]

I have just tried to use jaegertracing/jaeger-query:latest docker image and set base-path config option for query component. It seems this option works but there is error in browser console:

main.7cd7ea67.js:1 Uncaught SyntaxError: Unexpected token <

Browser: Chrome 63.0 Linux

[yurishkuro]

@maxkondr is this when loading the home page or some other view?

When I run the code locally from master I am not getting any errors

$ go run ./cmd/standalone/main.go --span-storage.type memory --query.base-path /jaeger

cc @tiffon

[tiffon]

@maxkondr I ran the latest query service in isolation via the following command:

docker run -d -p16686:16686 jaegertracing/jaeger-query:latest /go/bin/query-linux --query.static-files=/go/jaeger-ui/ --span-storage.type memory --query.base-path /jaeger

And was able to visit http://localhost:16686/jaeger without issue. The assets were loaded with the /jaeger prefix and the data calls had the /jaeger/api prefix and returned JSON data (albeit empty) as expected.

The error you're seeing suggests the query-service is not recognizing the prefix and is serving the fallback content (the index.html).

Can you inspect the HTML to see if there is a <base> element with an href attribute set to the base-path you configured?

<base href="/jaeger/" data-inject-target="BASE_URL">

[maxkondr]

@yurishkuro @tiffon Sorry for lack of info.

The error appears when I try to open home page (and the same for /search). Today I have tried to use 1.3 docker image but with no luck. My configuration looks like

apiVersion: v1
data:
  agent: |
    collector:
      host-port: "jaeger-collector:14267"
  collector: |
    es:
      server-urls: http://elasticsearch.logging:9200
    collector:
      zipkin:
        http-port: 9411
  query: |
    es:
      server-urls: http://elasticsearch.logging:9200
    query:
      static-files: /go/jaeger-ui/
      base-path: /tracing/jaeger
  span-storage-type: elasticsearch
kind: ConfigMap
metadata:
  labels:
    app: jaeger
    jaeger-infra: configuration
  name: jaeger-configuration
  namespace: tracing

Deployment:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
  labels:
    app: jaeger
    jaeger-infra: query-deployment
  name: jaeger-query
  namespace: tracing
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jaeger
      jaeger-infra: query-pod
  template:
    metadata:
      labels:
        app: jaeger
        jaeger-infra: query-pod
    spec:
      containers:
      - command:
        - /go/bin/query-linux
        - --config-file=/conf/query.yaml
        env:
        - name: SPAN_STORAGE_TYPE
          valueFrom:
            configMapKeyRef:
              key: span-storage-type
              name: jaeger-configuration
        image: jaegertracing/jaeger-query:1.3
        name: jaeger-query
        ports:
        - containerPort: 16686
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /
            port: 16686
            scheme: HTTP
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /conf
          name: jaeger-configuration-volume
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/role: minion
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
      volumes:
      - configMap:
          defaultMode: 420
          items:
          - key: query
            path: query.yaml
          name: jaeger-configuration
        name: jaeger-configuration-volume

Service:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: jaeger
    jaeger-infra: query-service
  name: jaeger-query
  namespace: tracing
spec:
  ports:
  - name: jaeger-query
    port: 80
    protocol: TCP
    targetPort: 16686
  selector:
    jaeger-infra: query-pod
  sessionAffinity: None
  type: ClusterIP

Ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/add-base-url: "true"
    nginx.ingress.kubernetes.io/from-to-www-redirect: "true"
    nginx.ingress.kubernetes.io/rewrite-target: /
  name: jaeger
  namespace: tracing
spec:
  rules:
  - host: etsys-sm-107.vms
    http:
      paths:
      - backend:
          serviceName: jaeger-query
          servicePort: jaeger-query
        path: /tracing/jaeger
  tls:
  - hosts:
    - etsys-sm-107.vms
    secretName: tracing-certs

The all HTML of home page is following:

<!DOCTYPE html>
<html lang="en">
<head>
<base href="https://etsys-sm-107.vms/tracing/jaeger">
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta http-equiv="cache-control" content="max-age=0"/>
<meta http-equiv="cache-control" content="no-cache"/>
<meta http-equiv="expires" content="0"/>
<meta http-equiv="expires" content="Tue, 01 Jan 1980 1:00:00 GMT"/>
<meta http-equiv="pragma" content="no-cache"/>
<base href="/tracing/jaeger/" data-inject-target="BASE_URL"/>
<link rel="shortcut icon" href="./favicon.ico">
<title>Jaeger UI</title>
<script>// Jaeger UI config data is embedded by the query-service. This is
// later merged with defaults into the redux `state.config` via
// src/utils/config/get-config.js. The default provided by the query
// service should be an empty object or it can leave `DEFAULT_CONFIG`
// unchanged.
function getJaegerUiConfig() {
const DEFAULT_CONFIG = null;
const JAEGER_CONFIG = DEFAULT_CONFIG;
return JAEGER_CONFIG;
}</script>
<link href="./static/css/main.160e898e.css" rel="stylesheet">
</head>

<body>
<div id="jaeger-ui-root"></div>
<script type="text/javascript" src="./static/js/main.771630f5.js">
</script>
</body>
</html>

Logs:

kubectl logs -n tracing jaeger-query-7479ddfd6-jzpc7
{"level":"info","ts":1522221188.9401312,"caller":"healthcheck/handler.go:99","msg":"Health Check server started","http-port":16687,"status":"unavailable"}
{"level":"info","ts":1522221189.0617292,"caller":"query/main.go:181","msg":"Archive storage not created","reason":"Archive storage not supported"}
{"level":"info","ts":1522221189.0627263,"caller":"query/main.go:128","msg":"Registering metrics handler with HTTP server","route":"/metrics"}
{"level":"info","ts":1522221189.0630171,"caller":"healthcheck/handler.go:133","msg":"Health Check state change","status":"ready"}
{"level":"info","ts":1522221189.0631068,"caller":"query/main.go:137","msg":"Starting jaeger-query HTTP server","port":16686}

If I run locally

docker run -d -p16686:16686 jaegertracing/jaeger-query:latest /go/bin/query-linux --query.static-files=/go/jaeger-ui/ --span-storage.type memory --query.base-path /jaeger

everything works fine

[maxkondr]

Seems I have found the root of the issue. I have removed next annotation from ingress:

nginx.ingress.kubernetes.io/rewrite-target: /

and now it works. thanks for help. you may close the TT.