search

zalando / zalenium

A flexible and scalable container based Selenium Grid with video recording, live preview, basic auth & dashboard.
https://opensource.zalando.com/zalenium/
Other
2.39k stars 574 forks source link

ERROR d.z.e.z.proxy.AutoStartProxySet #858

Closed maauso closed 5 years ago

maauso commented 5 years ago

🐛 Bug Report

Hi guys!

Deploying Zalenium with helm on EKS 1.11, we found a problem with proxy.AutoStartProxySet

Starting Nginx reverse proxy...
Starting Selenium Hub...
..11:01:43.258 [main] INFO  o.o.grid.selenium.GridLauncherV3 - Selenium server version: 3.141.59, revision: unknown
.11:01:43.328 [main] INFO  o.o.grid.selenium.GridLauncherV3 - Launching Selenium Grid hub on port 4445
.11:01:43.503 [main] INFO  d.z.e.z.c.k.KubernetesContainerClient - Initialising Kubernetes support
.11:01:43.852 [main] INFO  d.z.e.z.c.k.KubernetesContainerClient - Kubernetes support initialised.
        Pod name: zalenium-5d6bb95596-nvsc4
        app label: zalenium
        zalenium service name: zalenium
        Selenium Pod Resource Limits: {memory=Quantity(amount=2Gi, format=null, additionalProperties={}), cpu=Quantity(amount=<nil>, format=null, additionalProperties={})}
        Selenium Pod Resource Requests: {memory=Quantity(amount=2Gi, format=null, additionalProperties={}), cpu=Quantity(amount=1000m, format=null, additionalProperties={})}
11:01:43.852 [main] INFO  d.z.e.z.c.k.KubernetesContainerClient - About to clean up any left over docker-selenium pods created by Zalenium
11:01:43.894 [AutoStartProxyPoolPoller] INFO  d.z.e.z.proxy.AutoStartProxySet - Starting poller.
.11:01:43.973 [AutoStartProxyPoolPoller] ERROR d.z.e.z.proxy.AutoStartProxySet - Failed checking containers.
io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://172.20.0.1/api/v1/namespaces/default/pods. Message: Pod in version "v1" cannot be handled as a Pod: v1.Pod.Spec: v1.PodSpec.Containers: []v1.Container: v1.Container.Resources: v1.ResourceRequirements.Requests: Limits: unmarshalerDecoder: quantities must match the regular expression '^([+-]?[0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$', error found in #10 byte of ...|u":"<nil>"},"request|..., bigger context ...|resources":{"limits":{"memory":"2Gi","cpu":"<nil>"},"requests":{"memory":"2Gi","cpu":"1000m"}},"volu|.... Received status: Status(apiVersion=v1, code=400, details=null, kind=Status, message=Pod in version "v1" cannot be handled as a Pod: v1.Pod.Spec: v1.PodSpec.Containers: []v1.Container: v1.Container.Resources: v1.ResourceRequirements.Requests: Limits: unmarshalerDecoder: quantities must match the regular expression '^([+-]?[0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$', error found in #10 byte of ...|u":"<nil>"},"request|..., bigger context ...|resources":{"limits":{"memory":"2Gi","cpu":"<nil>"},"requests":{"memory":"2Gi","cpu":"1000m"}},"volu|..., metadata=ListMeta(_continue=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=BadRequest, status=Failure, additionalProperties={}).
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:476)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:415)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:381)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:344)
        at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleCreate(OperationSupport.java:227)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleCreate(BaseOperation.java:780)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:349)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation$1.apply(BaseOperation.java:366)
        at io.fabric8.kubernetes.api.model.DoneablePod.done(DoneablePod.java:26)
        at de.zalando.ep.zalenium.container.kubernetes.KubernetesContainerClient.createContainer(KubernetesContainerClient.java:330)
        at de.zalando.ep.zalenium.proxy.DockeredSeleniumStarter.startDockerSeleniumContainer(DockeredSeleniumStarter.java:279)
        at de.zalando.ep.zalenium.proxy.DockeredSeleniumStarter.startDockerSeleniumContainer(DockeredSeleniumStarter.java:248)
        at de.zalando.ep.zalenium.proxy.AutoStartProxySet.startContainer(AutoStartProxySet.java:446)
        at de.zalando.ep.zalenium.proxy.AutoStartProxySet.checkContainers(AutoStartProxySet.java:380)
        at de.zalando.ep.zalenium.proxy.AutoStartProxySet.lambda$0(AutoStartProxySet.java:124)
        at java.lang.Thread.run(Thread.java:748)
.11:01:44.245 [main] INFO  d.z.ep.zalenium.aspect.HubAspect - Registering custom Zalenium servlets
11:01:44.307 [main] INFO  org.openqa.grid.web.Hub - Selenium Grid hub is up and running
11:01:44.307 [main] INFO  org.openqa.grid.web.Hub - Nodes should register to http://10.11.15.104:4445/grid/register/
11:01:44.307 [main] INFO  org.openqa.grid.web.Hub - Clients should connect to http://10.11.15.104:4445/wd/hub
Selenium Hub started!
Sauce Labs not enabled...
Browser Stack not enabled...
TestingBot not enabled...
Zalenium is now ready!

We just modified values file adding persistent storage options.

To Reproduce

helm upgrade -i zalenium .  -f values.yaml

Expected behavior

To have an available agent

Environment

Zalenium Image Version(s): dosel/zalenium:3 , elgalu/selenium If using Kubernetes, specify your environment, and if relevant your manifests:

helm upgrade -i zalenium .  -f values.yaml --dry-run --debug
[debug] Created tunnel using local port: '33285'

[debug] SERVER: "127.0.0.1:33285"

REVISION: 2
RELEASED: Tue Feb  5 13:43:55 2019
CHART: zalenium-0.0.1
USER-SUPPLIED VALUES:
hub:
  basicAuth:
    enabled: false
    password: password
    username: zalenium
  browserStackEnabled: false
  browserStackKey: blank
  browserStackUser: blank
  cpuRequest: 1000m
  debugEnabled: false
  desiredContainers: 2
  env: null
  image: dosel/zalenium
  keepOnlyFailedTests: false
  maxDockerSeleniumContainers: 10
  maxTestSessions: 1
  memLimit: 2Gi
  memRequest: 2Gi
  newSessionsWaitTimeout: 600000
  openshift:
    deploymentConfig:
      enabled: false
      triggers:
      - type: ConfigChange
  port: 4444
  pullPolicy: IfNotPresent
  readinessTimeout: 1
  resources:
    limits:
      memory: 2Gi
    requests:
      cpu: 500m
      memory: 2Gi
  retentionPeriod: 3
  sauceAccessKey: blank
  sauceLabsEnabled: false
  sauceUserName: blank
  screenHeight: 900
  screenWidth: 1440
  securityContext:
    enabled: false
    fsGroup: 0
    runAsUser: 1001
  seleniumImageName: elgalu/selenium
  sendAnonymousUsageInfo: false
  serviceSessionAffinity: None
  serviceSourceRanges:
  - 0.0.0.0/0
  serviceType: NodePort
  tag: "3"
  testingBotEnabled: false
  testingBotKey: blank
  testingBotSecret: blank
  timeZone: UTC
  videoRecordingEnabled: true
ingress:
  annotations: null
  enabled: false
  hostname: zalenium.foobar.com
  path: /
  tls: false
nodeSelector:
  enabled: false
persistence:
  data:
    accessMode: ReadWriteOnce
    enabled: true
    name: null
    size: 50Gi
    storageClass: standard
    useExisting: false
  video:
    accessMode: ReadWriteOnce
    enabled: true
    name: null
    size: 50Gi
    storageClass: standard
    useExisting: false
rbac:
  clusterWideAccess: true
  create: true
serviceAccount:
  create: true
  name: null
tolerations:
  enabled: false

COMPUTED VALUES:
hub:
  basicAuth:
    enabled: false
    password: password
    username: zalenium
  browserStackEnabled: false
  browserStackKey: blank
  browserStackUser: blank
  cpuRequest: 1000m
  debugEnabled: false
  desiredContainers: 2
  env: null
  image: dosel/zalenium
  keepOnlyFailedTests: false
  maxDockerSeleniumContainers: 10
  maxTestSessions: 1
  memLimit: 2Gi
  memRequest: 2Gi
  newSessionsWaitTimeout: 600000
  openshift:
    deploymentConfig:
      enabled: false
      triggers:
      - type: ConfigChange
  port: 4444
  pullPolicy: IfNotPresent
  readinessTimeout: 1
  resources:
    limits:
      memory: 2Gi
    requests:
      cpu: 500m
      memory: 2Gi
  retentionPeriod: 3
  sauceAccessKey: blank
  sauceLabsEnabled: false
  sauceUserName: blank
  screenHeight: 900
  screenWidth: 1440
  securityContext:
    enabled: false
    fsGroup: 0
    runAsUser: 1001
  seleniumImageName: elgalu/selenium
  sendAnonymousUsageInfo: false
  serviceSessionAffinity: None
  serviceSourceRanges:
  - 0.0.0.0/0
  serviceType: NodePort
  tag: "3"
  testingBotEnabled: false
  testingBotKey: blank
  testingBotSecret: blank
  timeZone: UTC
  videoRecordingEnabled: true
ingress:
  annotations: null
  enabled: false
  hostname: zalenium.foobar.com
  path: /
  tls: false
nodeSelector:
  enabled: false
persistence:
  data:
    accessMode: ReadWriteOnce
    enabled: true
    name: null
    size: 50Gi
    storageClass: standard
    useExisting: false
  video:
    accessMode: ReadWriteOnce
    enabled: true
    name: null
    size: 50Gi
    storageClass: standard
    useExisting: false
rbac:
  clusterWideAccess: true
  create: true
serviceAccount:
  create: true
  name: null
tolerations:
  enabled: false

HOOKS:
MANIFEST:

---
# Source: zalenium/templates/pvc-data.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: "zalenium-data"
  labels:
    role: grid
    app: zalenium
    chart: zalenium-0.0.1
    release: zalenium
    heritage: Tiller
    app.kubernetes.io/name: zalenium
    helm.sh/chart: zalenium-0.0.1
    app.kubernetes.io/instance: zalenium
    app.kubernetes.io/managed-by: Tiller
spec:
  accessModes:
    - "ReadWriteOnce"
  resources:
    requests:
      storage: "50Gi"
  storageClassName: "standard"
---
# Source: zalenium/templates/pvc-video.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: "zalenium-videos"
  labels:
    role: grid
    app: zalenium
    chart: zalenium-0.0.1
    release: zalenium
    heritage: Tiller
    app.kubernetes.io/name: zalenium
    helm.sh/chart: zalenium-0.0.1
    app.kubernetes.io/instance: zalenium
    app.kubernetes.io/managed-by: Tiller
spec:
  accessModes:
    - "ReadWriteOnce"
  resources:
    requests:
      storage: "50Gi"
  storageClassName: "standard"
---
# Source: zalenium/templates/service-account.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: zalenium
  labels:
    role: grid
    app: zalenium
    chart: zalenium-0.0.1
    release: zalenium
    heritage: Tiller
    app.kubernetes.io/name: zalenium
    helm.sh/chart: zalenium-0.0.1
    app.kubernetes.io/instance: zalenium
    app.kubernetes.io/managed-by: Tiller
---
# Source: zalenium/templates/role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: "ClusterRole"
metadata:
  name: zalenium
  labels:
    role: grid
    app: zalenium
    chart: zalenium-0.0.1
    release: zalenium
    heritage: Tiller
    app.kubernetes.io/name: zalenium
    helm.sh/chart: zalenium-0.0.1
    app.kubernetes.io/instance: zalenium
    app.kubernetes.io/managed-by: Tiller
rules:
- apiGroups:
  - ""
  attributeRestrictions: null
  resources:
  - pods
  verbs:
  - create
  - delete
  - deleteCollection
  - get
  - list
  - watch
- apiGroups:
  - ""
  attributeRestrictions: null
  resources:
  - pods/exec
  verbs:
  - create
  - get
---
# Source: zalenium/templates/role-binding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: "ClusterRoleBinding"
metadata:
  name: zalenium
  labels:
    role: grid
    app: zalenium
    chart: zalenium-0.0.1
    release: zalenium
    heritage: Tiller
    app.kubernetes.io/name: zalenium
    helm.sh/chart: zalenium-0.0.1
    app.kubernetes.io/instance: zalenium
    app.kubernetes.io/managed-by: Tiller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: "ClusterRole"
  name: zalenium
subjects:
- kind: ServiceAccount
  name: zalenium
  namespace: "default"
---
# Source: zalenium/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: zalenium
  labels:
    role: grid
    app: zalenium
    chart: zalenium-0.0.1
    release: zalenium
    heritage: Tiller
    app.kubernetes.io/name: zalenium
    helm.sh/chart: zalenium-0.0.1
    app.kubernetes.io/instance: zalenium
    app.kubernetes.io/managed-by: Tiller
spec:
  type: "NodePort"
  sessionAffinity: "None"
  ports:
  - name: hub
    port: 4444
    targetPort: 4444
  selector:
    app.kubernetes.io/name: zalenium
    app.kubernetes.io/instance: zalenium
---
# Source: zalenium/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: zalenium
  labels:
    role: grid
    app: zalenium
    chart: zalenium-0.0.1
    release: zalenium
    heritage: Tiller
    app.kubernetes.io/name: zalenium
    helm.sh/chart: zalenium-0.0.1
    app.kubernetes.io/instance: zalenium
    app.kubernetes.io/managed-by: Tiller
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: zalenium
      app.kubernetes.io/instance: zalenium
  template:
    metadata:
      labels:
        role: grid
        app: zalenium
        chart: zalenium-0.0.1
        release: zalenium
        heritage: Tiller
        app.kubernetes.io/name: zalenium
        helm.sh/chart: zalenium-0.0.1
        app.kubernetes.io/instance: zalenium
        app.kubernetes.io/managed-by: Tiller
      annotations:
    spec:
      volumes:
        - name: zalenium-videos
          persistentVolumeClaim:
            claimName: zalenium-videos
        - name: zalenium-data
          persistentVolumeClaim:
            claimName: zalenium-data
      containers:
        - name: zalenium
          image: "dosel/zalenium:3"
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 4444
              protocol: TCP
          livenessProbe:
            httpGet:
              path: /grid/console
              port: 4444
            initialDelaySeconds: 90
            periodSeconds: 5
          readinessProbe:
            httpGet:
              path: /grid/console
              port: 4444
            timeoutSeconds: 1
          env:
            - name: ZALENIUM_KUBERNETES_CPU_REQUEST
              value: "1000m"
            - name: ZALENIUM_KUBERNETES_CPU_LIMIT
              value: "<nil>"
            - name: ZALENIUM_KUBERNETES_MEMORY_REQUEST
              value: "2Gi"
            - name: ZALENIUM_KUBERNETES_MEMORY_LIMIT
              value: "2Gi"
            - name: DESIRED_CONTAINERS
              value: "2"
            - name: MAX_DOCKER_SELENIUM_CONTAINERS
              value: "10"
            - name: SELENIUM_IMAGE_NAME
              value: "elgalu/selenium"
            - name: VIDEO_RECORDING_ENABLED
              value: "true"
            - name: SCREEN_WIDTH
              value: "1440"
            - name: SCREEN_HEIGHT
              value: "900"
            - name: MAX_TEST_SESSIONS
              value: "1"
            - name: NEW_SESSION_WAIT_TIMEOUT
              value: "600000"
            - name: DEBUG_ENABLED
              value: "false"
            - name: SEND_ANONYMOUS_USAGE_INFO
              value: "false"
            - name: TZ
              value: "UTC"
            - name: KEEP_ONLY_FAILED_TESTS
              value: "false"
            - name: RETENTION_PERIOD
              value: "3"
            - name: CONTEXT_PATH
              value: "/"
          args:
            - start
          resources:
            limits:
              memory: 2Gi
            requests:
              cpu: 500m
              memory: 2Gi

          volumeMounts:
            - name: zalenium-videos
              mountPath: /home/seluser/videos
            - name: zalenium-data
              mountPath: /tmp/mounted
      serviceAccountName: zalenium
Release "zalenium" has been upgraded. Happy Helming!
LAST DEPLOYED: Tue Feb  5 12:00:55 2019
NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1/Pod(related)
NAME                       READY  STATUS   RESTARTS  AGE
zalenium-5d6bb95596-nvsc4  1/1    Running  0         103m

==> v1/PersistentVolumeClaim
NAME             STATUS  VOLUME                                    CAPACITY  ACCESS MODES  STORAGECLASS  AGE
zalenium-data    Bound   pvc-4f719c96-2935-11e9-9bf9-120fe0989828  50Gi      RWO           standard      103m
zalenium-videos  Bound   pvc-4f72731e-2935-11e9-9bf9-120fe0989828  50Gi      RWO           standard      103m

==> v1/ServiceAccount
NAME      SECRETS  AGE
zalenium  1        103m

==> v1/ClusterRole
NAME      AGE
zalenium  103m

==> v1/ClusterRoleBinding
NAME      AGE
zalenium  103m

==> v1/Service
NAME      TYPE      CLUSTER-IP      EXTERNAL-IP  PORT(S)         AGE
zalenium  NodePort  172.20.237.182  <none>       4444:30886/TCP  103m

==> v1/Deployment
NAME      DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
zalenium  1        1        1           1          103m

NOTES:
###################################################
#                     Zalenium                    #
###################################################

Your release is named "zalenium".

To learn more about the release, try:

  $ helm status zalenium
  $ helm get zalenium
diemol commented 5 years ago

@maauso what happens if you don't change the values at all, does it work?

maauso commented 5 years ago

Hi, yes, without modifications works fine, I will try to more test on my values file until know what was the problem I close this ticket, thanks!