jaegertracing / jaeger-operator

Jaeger Operator for Kubernetes simplifies deploying and running Jaeger on Kubernetes.
https://www.jaegertracing.io/docs/latest/operator/
Apache License 2.0
1.02k stars 344 forks source link

Cannot create hpa for collector #942

Closed G-Asura closed 4 years ago

G-Asura commented 4 years ago

I found that if the replicas of the collector are configured as empty, hpa will not be created.

code
jpkrohling commented 4 years ago

This specific code shouldn't have an influence on the HPA. In fact, I did a small demo a few days ago about this feature, and this is the YAML that I used:

apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
  name: tracing
spec:
  strategy: production
  collector:
    image: quay.io/jpkroehling/jaeger-collector:SprintDemo35
    minReplicas: 3
    maxReplicas: 10
    resources:
      limits:
        cpu: 400m
        memory: 512Mi
      requests:
        cpu: 100m
        memory: 128Mi
    options:
      collector.queue-size-memory: 64
  storage:
    type: elasticsearch
    options:
      es:
        server-urls: http://elasticsearch.observability.svc:9200

In fact, it's the other way around: when you do specify a Replica size, we skip the auto scaling feature: https://github.com/jaegertracing/jaeger-operator/blob/9d6319d0f0e828e2a1781d7d5f6a0369d15439c8/pkg/deployment/collector.go#L204-L207

If you don't see the HPA in place, share your Jaeger CR and I can check whether it's supposed to create a HPA.

G-Asura commented 4 years ago

@jpkrohling I have seen this code, the real meaning I want to express is that when the Jaeger Deployment is created, if the default value of Replicas is set to 1, then HPA does not take effect in K8S and the replicas always 1,this is the Jaeger CR that I used:

apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
  name: jaeger-prod
  namespace: jaeger
spec:
  strategy: production
  collector:
    minReplicas: 3
    maxReplicas: 5
    image: myRegistry/jaeger-collector:1.16.0
  storage:
    type: elasticsearch
    dependencies:
      image: myRegistry/spark-dependencies:latest
      resources:
        requests:
          memory: "512Mi"
          cpu: "500m"
        limits:
          memory: "1024Mi"
          cpu: "1000m"
    esIndexCleaner:
      image: myRegistry/jaeger-es-index-cleaner:1.16.0
      numberOfDays: 14
    esRollover:
      image: myRegistry/jaeger-es-rollover:1.16.0
    options:
      es:
        server-urls: myEsurl
    secretName: jaeger-secret
  labels:
    app: jaeger
  resources:
    requests:
      memory: "254Mi"
      cpu: "250m"
    limits:
      memory: "512Mi"
      cpu: "500m"
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: myJaegerLabel
            operator: In
            values:
            - "yes"
    podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - jaeger
              topologyKey: kubernetes.io/hostname
jpkrohling commented 4 years ago

I just tried a YAML based on yours, and it seems to work for me:

$ kubectl get jaegers 
NAME      STATUS    VERSION   STRATEGY     STORAGE         AGE
tracing   Running   1.17.0    production   elasticsearch   65s

$ kubectl get hpa
NAME                REFERENCE                      TARGETS                        MINPODS   MAXPODS   REPLICAS   AGE
tracing-collector   Deployment/tracing-collector   <unknown>/90%, <unknown>/90%   3         5         1          16s

Here's the YAML I used for this test:

apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
  name: tracing
spec:
  strategy: production
  collector:
    minReplicas: 3
    maxReplicas: 5
  storage:
    type: elasticsearch
    options:
      es:
        server-urls: http://elasticsearch.default.svc:9200
  resources:
    requests:
      memory: "254Mi"
      cpu: "250m"
    limits:
      memory: "512Mi"
      cpu: "500m"
G-Asura commented 4 years ago

I solved this problem by upgrading to version 1.17.0,thanks!

jpkrohling commented 4 years ago

Indeed, HPA is available starting from 1.17.0. Glad you got it to work!